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I.  INTRODUCTION 


In  a recent 

SHARE 

study 

the 

annua  1 

growth  rate 

0 f 

software  aemands 

over  the  years 

1957 

- 1965  was  estimated 

t 0 

be  as  high  as 

21 

(15)  . 

The  same  st  udv 

i no i c a t e s 

that 

software  production 

rr  a v 

deve 1 oo 

at  a 

growth 

rate  of 

on 

ly 

1 1 .5-17%. 

Besides  this  guantitativ®  gao  # Boehm  [7]  ooints  out  the 
folio  wing  aual i tat i ve  deficiencies?  Since  software  is  in- 
creasingly used  for  important  functions  such  as  defense  sys- 
tems# traffic  control  svstems#  and  medical  ourposes# 
software  is  needed  which  can  be  trusted.  Although  SOX  of 
the  money  used  for  the  production  of  the  Apollo  banned 
Spaceflight  Program  was  devoted  to  testing  [131#  software 
errors  occurred  in  Apollo  8 , 11#  and  1 U . One  critical  error 
even  occurred  during  the  lunar  landing  phase. 

Software  engineering  (for  definition  see  Chacter  II)  may 
be  accepted  as  the  method  which  yields  the  answer  to  tne 
problem  of  producing  reliable  software  at  reasonable  cost 
(7).  Little  is  known  about  software  errors  besides  their 
existence  and  their  negative  influence  on  cost  and  perfor- 
mance. Increasing  difficulties  to  croduce  sufficient 
amounts  of  high  duality  software  make  it  mandatory  to  invent 
software  development  methods  which  are  designed  to  avoid  tne 
most  common  software  errors#  achieve  easier  testing  and 
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debugging  and  improve  methods  of  error  Detection. 

This  cannot  be  oone  without  oetailed  studies  about  the 
occurrence  of  software  errors.  Only  this  knowledge  will 
heio  to  identify  the  nature  of  errors  ana  finally  lead  to 
imorovements  of  software  methodoloay.  Other  motivating  fac- 
tors for  this  stuav  were: 

1.  Lack  of  currently  available  data  to  sucoort  analysis 

and  research  in  the  area  of  error  ocurrence  and 
detection. 

2 . Tne  need  to  examine  existing  software  error  models. 

The  objective  of  the  e«oeriment  .as  to  gather  accurate 
and  complete  information  about  errors/  their  occurrence  ana 
detection  throughout  the  entire  crocess  of  software  level co* 
ment.  Limitations  of  the  experiment  were  the  relatively 
small  number  of  projects  and  the  lack  of  experienced  pro- 
grammers. The  use  of  the  experimenter  as  the  supject  pro- 
grammer and  evaluator  introduced  biases  also.  The  scope  of 
the  experiment  did  net  include  the  oroqram  maintenance 
phase.  It  was  felt  to  be  of  oreat  importance  to  record  all 
errors/  even  trivial  errors  which  could  hurt  the  image  of 
the  programmer.  A detailed  description/  of  the  experiment 
and  the  error  data  is  given  in  chapters  4 thru  5 and  in  the 
append i c es  . 
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I I . concepts  and  keywords 

Communication  among  computer  scientists  is  hingerea  oy 
the  lack  of  common  terminoloay.  In  order  to  avoid  misunder- 
standing* the  most  important  keywords  will  be  defined  in 
this  chaoter.  wherever  possible  definitions  are  aaoDtea 
from  other  publications. 

A.  SOFTWARE  ENGINEERING  TERMS 

1.  Software  Engineering 

" I he  practical  application  of  scientific  kncwledae 
in  the  design  and  construction  of  computer  programs  and 
the  associated  doc u«en t a t i on  reouired  to  develop* 
operate  and  maintain  them",  f 7 1 


2 . Too- C own  Design 

A program  design  method  whicn  starts  at  a 
eral  level  cv  identifying  major  functions  and 
steowise  to  lower  levels  to  the  identification 
functions  that  derive  from  the  major  ones, 
design"  and  "hierarchical  program  design"  are 


very  gen- 
oroceeds 
of  lesser 
"Tco-down 
eou ivalent 
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terms,  t 1 2) 


3.  Structured  Proaramm i ng 

Although  this  ter»i  is  used  in  numerous  publications 
its  unde r s t and i ng  varies  Decause  oi  its  conceotual  na- 
ture. As  orooosec  bv  Dijkstra  as  early  as  1 9 b 5 (30),  it 

first  was  a concent  to  eliminate  GO  TO  statements. 
Later  on  it  was  combined  with  the  idea  of  too-down 
design.  [311 

Rather  than  orovidina  a formal  definition/  Yourdan 
(12)  and  Auerbach  12 7]  describe  structured  programming 
by  its  objectives:  "increase  of  readability"  ana  "de- 
crease of  testing  problems."  Tne  following  definition 
of  structured  orocrammina  as  it  applies  to  this  paper  is 
derived  from  Ki nchgaessnpr ' s description.  (281 

Structured  programming  is  a method  of  cevelooing 
programs  for  which  the  underlying  concept  is  a too-down 
design  which  eventually  leads  to  a modular  structure  of 
the  Program.  All  algorithms  are  designed  usinq  the  five 
major  constructs  allowed  in  structured  programming: 

- Seguentia!  statement 

- IF  THEM  ELSE  statement 

- .vhILE  Ioop  (conaitional  looo) 

- FOR  looo  (iterative  looo) 

- CASE  statement 

In  addition  to  t*is  definition  it  should  oe 
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emphasized  that  structured  programming  is  a design  dis- 
cipline rather  than  a »ay  of  writing  code.  Therefore 
it  should  not  oe  defined  as  "coding  w i t n o u t GO  TO 
statements."  Methoos  have  oeen  deve'oced  to  implement 
structured  designs  in  FOPTPAD.  (121  However/  it  is 
easier,  ana  more  readable  to  use  a bloc*  structured 
lanauaqe,  such  as  ALGOL,  to  imolement  a structured 
des  i qn  . 

A.  Directed  Grac" 

"A  directed  graph  is  a qecmetric  qrach , consisting 
of  nodes  and  arcs  with  a direction  of  traversal  associ- 
ated with  each  arc."  (lb) 

5 . Modu 1 e 

"A  module  is  a physical  compination  of  program 
instructions  that  is  independent  of  others  with  rescect 
to  compiling,  assembling  and  loaaina  and  which  performs 
a specific  function.”  [16]  "Each  module  has  a small 
number  of  interactions  with  other  modules."  [4] 

6.  Pro dram 

A proaram  is  a set  of  integrated  modules.  (In) 

7.  Software  Development  Process 

The  software  development  process  includes  the 
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following  st  ane s : 

- Proolem  Analysis 

- Design  and  Desigr  Review 

- Coding 

- Debuggi no 

- Testing 

- Inteoration 

- I *o 1 enen t a t i on 

“ Maintenance 

Desoite  minor  differences  with  o t h e r publications 
such  as  Aueroaeh  tc?)  the  above  view  of  the  software 
development  orocess  » i 1 1 oe  adopted  as  the  definition  of 
the  software  development  process  for  this  thesis. 
Althouoh  the  development  staaes  are  listed  seauentialtv 
it  should  be  noted  that  software  production  in  general 
is  viewed  as  a dynamic  process  including  feedbacks  to 
earlier  stages  of  a project.  Thus  none  of  these  phases 
mav  be  considered  to  be  completed  when  the  subseauent 
phase  has  starteo.  In  some  projects#  testing  itself  is 
preceded  oy  activities  such  as  the  writing  of  test 
specifications  ana  test  procedures.  Fiaure  1 indicates 
nossiole  feedback  situations  within  a complex  program 
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FIGURE  1 

The  FEE  DRACh  CYCLES  DURING  A SOFTWARE  DEVELOPMENT  PROJECT 


A . Common  Data 

Common  Data  is  data  which  is  used  by  several 
nodu 1 e s . 


a - 


-J 


R.  TESTING,  DEBUGGING  AND  ERRORS 
1 . Debuag i no 

(a)  Debugging  is  the  action  to  check  subroutines  and 
nodules  as  to  whether  or  not  they  perform  according 
to  Programmer  exoec t at i ons  . It  is  also  the  action 
which  one  takes  to  locate  ana  correct  known  errors. 

(b)  festina  is  the  action  taken  after  modules  have 
been  intearated  to  check  whether  or  not  the  oroaram 
meets  soec  i f i c at  i ons  . As  statec  bv  Cijkstra  , 
testing  can  only  estaolish  the  crese^ce  of  errors, 
not  their  absence. 

<?.  Software  Error 

A software  error  is  a mistake  maae  during  sett  ware 
develooment,  which  leads  to  an  ircorrect  action  or 
result  with  resoect  to  the  oroaram  soec i f i c a t i ons  within 
the  program.  IloJ  Errors  made  in  the  soeciti cation 
ohase  ot  the  oroaram,  haraware  errors  and  compiler  er- 
rors (i.e.  errors  within  the  compiler  program)  were  not 
studied  durina  this  exoeriment.  Cateaor i zat i on  and 
definitions  of  software  errors  with  respect  to  their 
occurrence  are  aiven  in  cnaoter  3. 
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C.  SOFTWARE  RELIABILITY  A NO  CORRECTNESS 


1.  Software  Reliability 


"Software  reliability  is  t^e  probability  that  a com- 
puter program  will  perform  its  (user)  intended  function 
for  a specified  time  interval  under  stated  operating 
conditions."  [17) 


2.  Software  Correctness 

"Ihe  feature  of  software  that  renders  it  operation- 
ally useful  for  its  intended  functions  is  its  correct- 
ness. Correct  software  does  the  thinas  intended  by 


aes ion."  [q] 


r 


III.  DEFINITION  OF  ERROR  CATEGORIES  AfgO  TYPES 


Y«hen  analyzing  the  causes  of  errors  in  software  prortuc- 

x 

tion  there  is  an  obvious  need  to  orecisely  describe  an  error 
in  accordance  with  its  category  and  tyce.  One  of  the  objec- 
tives of  this  study  was  to  identify  errors  with  resoect  to 
their  occurrences  ana  causes. 

It  was  was  found  that  many  errors  in  software  oroduction 
belong  to  one  of  the  following  major  categories: 

1.  Problem  Specification  Errors 

2.  System  Design  Errors 

3.  Droaram  Design  Errors 

4 . Coding  Errors 

5.  Clerical  Errors 
o.  Debugai no  Errors 
7.  Testing  Errors 

Implementation  Errors 


A.  0 E F I N I T U N S OF  ERRGR  CATEGORIES 


l . Problem  Specification  Error 

Any  mistake  or  deficiency  w 
analysis  of  the  proara-  application 
software  r«aui regents  with  resoect 


a s 


n i c * 

occurs  i 

n 

the 

o r i 

i n spec i f v i 

ng 

the 

to  the  intended  p r o - 

C r 


gram 


application  such 


i ncorol e t e 


erroneous 


amoiquous  statements. 

2.  System  design  Error 

An  error  maae  in  the  transformation  of  the  user- 
originated  proqrarr  soec  i f i c a t i on  s into  systems  design 
spec i f i cat i ons  . 

3.  Program  Design  Error 

A program  desiqn  error  is  one  made  in  the  transfor- 
mation of  the  system  desiqn  into  specific  alaorithms  ana 
data  structures. 

y.  Coding  Error 

A coding  error  is  one  which  is  maae  auring  the 
transformation  of  a proaram  desiqn  into  source  1 anouaae. 

5.  Clerical  Error 

A clerical  error  is  Human  failure  which  is  made  dur- 
ing the  t r an s f o r ma t i on  of  any  onysical  rec resen t a t i on 
(such  as  a coding  sheet)  of  the  oroaram  or  carts  of  the 
proaram  to  another  physical  r ec r e s en t a t i on  (such  as 
Punch  cards). 

fc>.  Debudqing  Error 

Inaoorooriate  use  of  deruaaina  tools,  insufficient 
or  i naooropri atp  selection  of  test  cases > test  data  or 


- 1 « - 


-n  i s i n t e ro  re  t a t i on  of  aeouooi  nq  results. 

7 . Testing  Error 

InaoBroori  ate  or  insufficient  test  cases  or  test, 
data#  mi sunderstanding  of  functional  requirements  of  t^e 
program,  deviation  from  test  ol  an  or  mismteroretation 
of  test  results. 

8.  [itidI  ement  at  i or  Enror 

Unexpected  environmental  preplan  which  occurs  curing 
the  imolementaticn  of  a Program  which  could  not  have 
been  anticipated  in  a orevious  stage  of  ttie  software 
development.  This  includes  cases  such  as  changes  of 
hardware  by  the  manufacturer  and  cerfor-ance  problems 
with  ooeratina  systems. 

^ccorainq  to  h he  cefinition  of  software  errors,  an  error 
which  goes  not  lead  to  an  incorrect  action  or  result  within 
the  program  cannot  be  identified  as  a software  error.  For 
example  insufficient  test  cases  during  aebuaging  of  a 
subroutine  may  not  lead  to  a software  error.  however,  t^e 
potential  exists  for  undetected  software  errors  wren  test 
cases  are  insufficient.  According  to  the  'definition  of 
debuogina  errors/  errors  of  this  category  do  not  necessarily 
affect  the  prooram.  Therefore  oe pugging  errors  will  not  be 
oetectea  very  often;  however,  because  of  being  a potential 
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error  source*  errors  of  this  category  were  considered  to  be 


imDortant.  During  the  experiment  described  in  chapter  4 all 
deCuagi nq  errors  were  recorded.  Similarly  design  errors* 
which  were  detected  durinq  a design  reviewi'or  faulty  com- 
ments were  recorded.  This  was  of  great  importance*  since  it 
yields  significant  information  about  error  sources. 


B.  DEFINITION  OF  ERROR  TYPES 

The  list  of  error  tyre  definitions  given  below  identi- 
fies the  most  common  types  of  errors  within  the  major  ca- 
teaories  of  software  errors.  If  appropriate*  examples  are 
given  for  clarity. 

These  definitions  are  imoortant  because  it  has  been 
discovered  that  without  a definition  of  error  types*  errors 
of  tne  same  type  could  be  recorded  in  different  ways*  not 
only  by  different  programmers,  put  also  by  the  same  person 
at  different  times. 


1 . Des i on  Errors 

The  following  types  of  errors  apply  to  both  ca- 
teaories  "System  Design  Errors"  and  "Program  Design 
Errors": 


Di  : Communication  Error 


An  error  occurs  due  to  improper  communication 
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between  members  of  a programming  team  or  due  to 


i naoorop r i a t e svstems  desiqn  specifications.  Typi- 
cal cases  are  conflicting  i n t e rpre t a t i on  of  common 
data  ana  mi sunderst andi no  of  module  functions. 

De?  : Design  Neqliaence 

parts  of  proqram  or  svstem  aesiqn  soeci fcations  are 
neglected  or  necessary  cooperation  with  the 
oriainator(s)  of  the  specification  have  been  omit- 
ted. such  as  neglecting  desired  outout  formats  or 
lack  of  documentation. 

D3  : Forgotten  Cases  or  Stecs 

while  solving  the  problem  one  or  more  cases  have 
not  been  considered,  such  as  neglecting  leap  years 
for  calculation  of  dates  or  necessary  steoCs)  to 
solve  the  problem  have  been  omitted,  such  as  deter- 
mining the  status  of  peripheral  devices  before  open- 
ing I/C,  or  forgotten  conversion  from  decimal  to 
binary  or  vice  versa. 

04  : Timing  Problems 

Misconception  in  scheduling  program  events,  such  as 
assigning  1 naco roo r i a t e priorities  or  chanqinq  data 
before  completion  of  I/O. 

Db  : Errors  in  I/O  Concepts 

Errors  made  in  the  desian  of  input  or  output,  such 
as  m i sunde r s t ana i ng  of  hardware  reaui rements  of  I/O 
devices  or  channels.  Typical  cases  are 


m i sunae r s t ana i nq  of  the  capabilities  of  terminals  or 


t 


graphics  devices*  neglecting  the  size  of  lines  on  a 
1 ineprinter  or  the  interrupt  capaoilities  of  peri- 
pheral devices. 

Do  : Data  Design  Error 

The  data  design  does  not  fit  the  needs  of  the  pro- 
gram. Typical  examples  are  wrona  sizes  of  arrays* 
puffers  or  freelists.  Also  cases  of  inappropriate 
data  definitions*  such  thaf  the  chosen  aata  types  do 
not  accomplish  the  desired  orecision.  This  error 
tyoe  also  includes  design  of  inappropriate  I/O  for- 
mats and  improper  desiqn  of  common  data. 

D/  : I n i t i a I z a t i on  Error 

wrong  or  incomplete  initialization  such  as  forgotten 
i n i t i a 1 i z a t i on  of  global  pointers*  arrays  or  flag 
variables  or  for gotten  i n i t i a 1 i z a t i on  of  interrupt 
hand  I ers  . 

Do  : Inadequate  Checkina 

Checking  of  vari ac I es  or  input  data  is  incomplete  or 
wrong  according  to  program  specifications. 

Dv  : Extreme  Conditions  Neqlectea 

Data  or  machine  aeoendent  extreme  conditions  are 
nealected*  such  as  numeric  values  which  cause  un- 
derflow* overflow*  excess  of  array  limits  or  maximum 
values  of  real  numbers  or  integers*  ana  occurrence 
of  zero  or  negativ®  values  due  to  round  off. 
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DIO:  Seauencing  Error 

Program  events  or  oecisions  scheduled  in  wrong  se- 
quence such  as  transformation  from  problem  statement 
into  a sequence  of  oDerations. 

Dll:  Index i ng  Error 

Faulty  index  calculation  such  as  designing  an  er- 
roneous aloorithm  for  accessina  array  elements  or 
items  of  a table. 

D12:  Lood  Control  Errors 

Error  in  controlling  either  an  iterative  or  condi- 
tional reoetitive  algorithm,  such  as  basing  exit  or 
break  decisions  on  conditions  which  can  never  occur. 
D 1 3 : Misuse  of  Boolean  Exoression 

An  error  is  made  in  using  Boolean  operators,  such  as 
constructing  a false  inverse  of  an  expression.  A 
trivial  examole  of  this  error  type  is  solving 
(x+y)'  * z - ( x 1 tv')  * z 

instead  of: 

(x  + y 1 * * z - (x'  * y')  * z 

where  +,  * , ' are  the  Boolean  operators  "OF",  "AND", 

and  "NOT"  respectively. 

Did;  Mathematical  Error 

The  mathematical  solution  of  a problem  within  the 
programming  project  does  not  or  not  always  represent 
the  correct  result  according  to  the  proqram  specifi- 
cations. This  includes  cases  in  which  a 


21 


descriotions  or  critical  terms,  such  as  mathematical 


*1 


expressions  which  leads  to  an  error  in  iTole "menta- 
tion ot  the  underlying  a e s i an. 

C2  : hieo 1 i gence 

Parts  ol  the  design  are  neglected  or  necessary 
cooperation  with  the  designer(s)  is  omitted.  This 
will  include  cases  where  croorammers  are  uncertain 
about  the  design  or  oarts  of  it  and  fail  to  t a < e 
appropriate  action,  such  as  request i na  an  exclama- 
tion from  the  desicjner(s)  or  obtaining  the  missing 
information  from  other  sources  such  as  project 
specifications,  hardware  or  software  manuals. 

C3  : I/O  Format  Error 

Use  of  imorooer  I/O  formats  in  coding,  such  as  using 
formatted  I/O  when  nonformatted  I/O  is  soecifiec  or 
using  i naDoroori ate  record  lengths. 

C 4 : '"'i  solaced  Data  Declaration 

Error  made  in  declaring  data  either  in  tne  wrong 
block  level  (while  coding  in  a block  structured 
languaael  or  in  an  inanorooriate  oos i t i or  within  the 
source  c ode . 

C5  : Multiple  Data  Declaration 

M u I t i c I e declarations  of  same  data. 

C o : vi ssinq  Data  Declaration 

Necessary  data  declaration  left  out. 

C7  : Inaneauate  Data 
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Error  made  in  choosina  size  or  type  of  data*  such  as 
declaring  single  precision  instead  of  double  creci ■ 
s i on . 

C8  : Initialization  Error 

Wronq  or  forgotten  initialization. 

C9  : Error  in  Parameter  Passing 

Passing  of  parameters  is  incomplete,  wrong  or  types 
do  not  match. 

CIO:  Inadeauate  or  Forgotten  Checking 

Checking  of  inout  is  either  incomplete  or  wrong 
according  to  program  design  specifications,  such  as 
not  checktna  for  invalid  incut  within  a subroutine 
which  receives  characters  from  a user  terminal.  A 
typical  case  of  this  error  tyre  occurs  when  numeri- 
cal inout  is  expected  and  is  converted  to  binary 
without  check inq  the  validity  of  the  input  charac- 
ter. Another  case  is  an  inadeauate  check  of  the 
size  of  the  input  numoer . 

C 1 1 : Level  Problems 

nhile  codina  in  a bloc*  structured  language  an  error 
results  from  confusing  bloc*  levels.  such  as 
misunderstanding  the  scoce  of  variables,  e . g . an 
ALGOL  programmer  tries  to  use  variables  or  data  of 
a lower  level  within  the  main  level  (level  1)  of  a 
oroaram . 

C 1 C? : wi  ssinq  Declarations  of  31  o c k Limits 
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/I  h i 1 e coding  in  a d]  oc  k structured  language  mandato- 
ry declarations  of  blocs  level  limits  such  as  BEGIN  r 
END  are  left  cut. 

C13:  Case  selection  error 

An  error  is  made  in  connection  with  a CASE  statement 
or  comcuted  GO  TO  statement  or  an  equivalent  con- 
struct. For  example  in  an  i mo ) emen t a t i on  of  a GO  TO 
SwITCh  in  C V S - 2 an  error  occurs  due  to  wrong  se- 
quence of  labels.  CMS-2  is  a proaramming  languaae 
used  for  tactical  programming  within  military  aool i * 
cat i ons  (ibl . 

C 1 « : GO  TC  Problems 

An  error  is  made  in  connection  with  the  use  of  a GO 

TO  statement. 

CIS:  Comment  Error 

Statinq  a faulty  or  misleading  comment. 

C 1 h : Forqot  ten  Delimiter 

Delimiter  left  out. 

C 1 7 : Inconsistency  in  Naming 

An  error  is  made  when  naming  an  identifier  in  dif- 
ferent ways.  For  example  a subroutine  declared  as 
GETCHAR  is  called  by  using  GETCH. 

C 1 8 : Nrona  Use  of  Nested  IF  Statements 


A n 

error  is  ^aoe 

dur i no  the 

construction 

of  a nested 

IF 

s t a t enen t • 

Errors  of 

this  type 

occur 

i n 

transforming  a 

f 1 ow chart 

or  decision 

table 

logic 
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into  nested  IF  statements  or  using  an  incorrect  for- 
mat which  will  cause  a comgile  error  or  associating 
any  of  the  ELSE  cases  with  a wrong  IF  clause. 

C 1 9 : Indexing  Error 

Indexing  is  not  aoorooriate  or  left  out. 

C20;  Inconsistent  Use  of  Variables  or  Data 
An  error  occurs  because  variables  or  data  are  used 
for  conflicting  ourooses.  Fcr  examcle  usage  of  a 
temoorary  storage  location  in  different  Darts  of  the 
program  coulc  be  implemented  in  such  a manner  that 
information  is  overwritten  bv  some  subroutine  but 
still  be  needed  by  another  subroutine.  Another 
examole  is  th»  usage  of  common  variables  fcr  dif- 
ferent ourooses  like  pointer  and  index. 

C21:  Seouencing  Error 

An  improper  seauence  of  statements  which  leads  to  an 
error.  For  examole  an  exchange  of  array  elements  is 
D r oq  rammed 

TEMP  «-  A ( i ) 

A ( j ) «-  TE^P 
A ( i ) «-  A ( j ) 

which  does  not  accomplish  the  desired  exchange  of 
the  two  el emen  t s . 

C<?2:  Flag  usaae  Problems 

An  error  is  made  in  using  f 1 ags  in  order  to  control 
o r og r am  1 oa i c . 
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Syntax  Error* 


C23: 


Violation 

o * 

syntax  rules  of 

the  programming 

1 anguage . 

This  i 

nc 1 udes  errors  in 

connection  with 

transformation  of 

boolean  exoressi 

ons  and  arithmetic 

express i ons 

• 

C2<4:  Loop 

Con  t r o 

1 E r r o r 

Error  in  implementing  the  control  structures  of  an 
iterative  or  conditional  repetitive  alaorithm. 

C 2 5 : Incorrect  Exit  f rom  Subrout i nes 

ip  error  occurs  in  the  implementation  of  an  exit 
from  a subroutine. 

C2b : Language  usaoe  Problems 

in  error  is  made  due  to  idiosyncrasies  of  the  oro* 
dramming  lancuaae  which  are  not  thoroughly  under- 
stood by  the  cro q rammer. 

C <£  7 : Forgotten  Statements 

in  error  is  made  by  leaving  out  a necessary  state- 
ment which  the  oroorammer  knew  was  necessary. 

C28:  Represent  at i nn  Error 

i codino  error  which  is  introduced  during  the  Pro- 
cess of  the  physical  represent  at i on  of  thoughts^ 
such  as  writina  a statement  different  from  what  one 
i n t ended  . 

C29:  Control  Sequence  Error 

in  error  is  made  which  causes  the  program  to  follow 
a wrong  nranct  af t»r  a decision. 
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C30:  Incorrect  Subroutine  Usaqe 


Using  the  wrong  subroutine  or  using  a subroutine 
i n apo r cc r i a t e 1 y » e.q.  a programmer's  expectation  of 
what  a Subroutine  will  accomplish  is  different  from 
what  the  subroutine  actually  does. 

C 3 1 : Other  Cooinq  Errors 

iny  other  coding  error  which  is  is  not  one  of  the 
types  listed  above. 

3.  Clerical  Errors 

M : Manual  Error 

An  error  resultin q from  1 a c x of  motoric  skill  or 
temporary  manual  misfunction/  such  as 

- Errors  of  Commission  (e.g.  writing  or  typing 
"busu"  instead  of  "busy") 

- Errors  of  Omission  (e.g.  writing  or  typing 
"b'J.W*  instead  of  ”d»0'vN") 

- Errors  of  Transposition  (e.g.  writing  or  typing 
"hte"  instead  of  "the") 

A«?  : Cental  Error 

An  error  resulting  from  a " e n t a I misfunction*  such 

as 

- Perceptual  Errors  (e.q.  readino  "0"  irstead  of 
”0"*  II  instead  of  11  or  "DOG"  instead  of  "DOD") 
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- Expectation  Pryors  (e.g.  in  a given  context  a 
certain  word  or  expression  is  excected  ang  the 
actual  word  or  exoression  is  overlooked*  sued  as 
assuming  "FOR  I:=N  STEP  1 UNTIL  i*"  DC"  where  "FOR 
N : = 1 STEP  -1  UNTIL  M DO"  is  the  the  actual  im- 
plementation of  an  iterative  looo  in  ALGOL  ft.) 

A3  : Procedural  Errors 

Given  a procer  assianment  for  a clerical  task  which 
implies  some  well  defined  seauence  of  actions  which 
could  be  excected  from  the  assignment*  some  of  the 
steps  are  either  feraotten  or  not  carried  out  o r o o ~ 
erl y«  such  as  some  lines  of  coce  on  a coding  sheet 
are  omitted  by  t">e  keypunch  operator  or  cara  correc- 
t i on  procedures  are  not  followed  aoo r op r i a t e 1 y or 
inserted  cards  are  misplaced. 

Ay  : Other  Clerical  Errors 

Any  other  clerical  error  which  is  not  one  of  the 
types  listed  above. 

y.  Debuaginq  Errors 

61  : I napo roc r i at e Use  of  Deouaoing  Tools 

ftrong  selection  of  debugaino  tools*  such  as  traces* 
snapshots  or  aumos  or  using  them  inappropriately. 
For  example  using  a snapshot  oefore  instead  of  after 
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a certain  instruction  or  taking  a dump  of  wrong 
memory  locations. 

B2  : Insufficient  or  Inacorooriate  Selection  of 

Test  Cases  or  Test  Data 

The  test  cases  or  test  data  provide  insufficient 
test  coverage  of  t re  p roar  am  or  fail  to  provide 
testina  of  critical  oarts  of  the  oroaram. 

B 3 : N'isinteroretation  of  Debu  going  Hesults 

in  inaporoori ate  action  is  t a < e n due  to  misin- 
teroretation  of  debugging  results,  such  as  a wrong 
result  is  overlooked  or  m i auntie  r s t and  i ng  of  a 
correct  result  leads  to  additional  errors. 

Bit  : Misinterpretation  of  the  Error  Source 

in  inaDnrooriate  action  in  correct ina  a discovered 
software  error  that  results  f rot  wrong  assurot ions 


about 

the  error  source. 

such 

as  correcting 

the 

s yrro- 

toms 

of  an  error  rather 

than 

its  cause  or 

the 

D po- 

grander  fails  to  foresee  the  impact  of  a change  on 
other  parts  of  the  proarar  and  inserts  additional 
errors  into  t n e orogram, 

BS  : Neg 1 i gene  e 

Errors  resulting  from  negligence  in  debugging.  For 
ex  a mole  the  imglementation  of  an  error  correction  is 
assumed  to  be  ®rror  free  without  further  checking. 

B b : Other  Debugging  Errors 

inv  other  oePuoqi na  error  which  is  not  one  of  the 
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types  listed  aoove 


^ . Test  ina  Errors 

T1  : Inaoeauate  Test  Case(s)  or  Test  Data 

Selected  test  case(s)  or  test  data  do  not  orooerl y 
test  the  orogram. 

T l : Misinterpretation  of  Test  Results 

An  i nacp roo r i a t e action  is  ta*en  due  to  misunder- 
standing of  test  results,  sue*  as  a wrong  result  is 
overlooked  or  misunderstanding  of  a correct  result 
leads  to  an  implementation  of  additional  errors. 

T3  : Misinterpretation  of  Problem  Specifications 

Errcr(s)  resulting  from  m i s u n de r s t and i mg  the  program 
spec i f i c a t i ons , such  as  incorrect  interpretation  of 
user  defined  functions  which  leads  to  an  insertion 
of  an  error  or  to  over  looking  an  existing  error. 

Tu  : Negligence 

Errors  resultino  from  negligence  in  hestina.  For 
example  a tactical  real  time  svstem  is  merely  tested 
within  specified  limits  and  tests  fail  to  examine 
the  oehavior  of-  the  system  for  cases  where  limits 
are  exceded.  In  this  particular  example  negligence 
can  cause  serious  results  in  critical  situations. 
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T5  : 

Other 

Testing  Frrors 

Any 

other 

testing  error 

which 

is  not  one  of  the 

t yoe  s 

1 i s t ed 

above . 

In  order  to  get  meaninqful  results,  the  above  defini- 
tions were  used  throughout  the  croject.  This  list  is  not 
considered  complete.  however,  it  permittee  identification 
of  software  error  types  for  the  purpose  of  this  experiment. 
Some  of  the  listed  error  tvoes  could  be  further  subdivided 
into  subsets  of  each  type  if  this  were  desired. 

To  contribute  to  analysis  of  error  sources,  it  was 
decided  to  list  the  same  error  tvee  which  occurs  in  dif- 
ferent software  development  phases  under  different  ca- 
tegories. To r example  one  type  of  error  which  has  the  same 
name  (loop  control  error)  may  occur  in  both  design  and  cod- 
ing phases.  ho-ever,  the  cause  of  the  error  could  be  dif- 
ferent . 

Error  severity  is  related  to  the  effect  of  the  error  on 
system  operation.  There  is  no  direct  relationship  between 
error  tyre  ana  severity.  Tor  example  mistyping  of  a single 
character  of  a variable  name  in  FOtiTtfAN  can  cause  strange 
results,  whereas  the  same  error  made  in  a comment  mav  not 
affect  the  o roaram  at  all.  The  cons i oerat i on  of  error 
severity  was  oevond  the  scope  of  this  project. 
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I V . EXPERIMENT  DESCRIPTION 
4.  GENERAL  REMARKS 

Error  asta  were  gathered  in  four  programming  projects 
ranginq  f ron  snail  to  large.  During  280  man  hours  of  total 
project  time  more  than  2000  source  statements  were  produced. 
A total  of  173  errors  w e r e recorded. 

The  scooe  of  the  programming  projects  was  limited  to  1 8 e 
fo'lowina  software  development  phases: 

- Desian  and  Design  Review 

- Coding 

- Debuqq i ng 

- Testing 

8.  DESIGN  OP  TrtF  EXPERIMENT 

The  experiment  was  designed  to  guarantee  t^e  most  accu- 
rate recording  of  relevant  error  information.  Therefore* 
for  each  of  these  phases*  an  appropriate  form  was  designed 
to  qive  a firm  ou iceline  to  the  experiment  programmer  for 
recordina  the  entire  development  of  a proqram  including 
necessary  details  about  errors.  The  error  data  which  were 
recorded  on  these  forms  for  project  1*  2*  3 and  4 are  given 

in  Acoengices  A,  8,  C ang  0 respectively.  For  «*ach  project 
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the  recorded  data  was  analyzed  with  respect  to  the  following 
f actors : 

- Number  of  source  statements  oroduceo 

- wan  hours  scent  on  the  project 

- Man  hours  scent  in  each  of  the  software 
development  chases 

- CPU  time  used  for  compiles 

- CPU  time  used  for  test  and  debug  runs 

- Number  of  test  ana  debugging  steps  needed 

- Project  time  used  to  correct  errors 

- Project  time  oetween  error  detections 

- Occurrence  of  errors  with  respect  to  software 
development  phases 

- Detection  of  errors  with  resoect  to  software 
development  phases 

- Time  history  of  error  detection  and  correction 

- Complexity  measures  of  subroutines 

4 summary  of  this  analysis  is  documented  at  tne  end  of 
each  aocendi x . 

Since  it  would  be  too  difficult  to  obtain  error  data  in 

a commercial  environment*  it  was  neccessary  to  obtain  tne 
> 

data  by  using  a carefully  controlled  experiment  in  an 
academic  environment.  T*e  considerable  overhead  necessary 
to  record  all  relevant  data  was  not  included  in  project 
time.  It  orobanly  would  not  be  feasible  to  conduct  this 
kind  of  experiment  in  a commercial  environment  because: 
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- The  overhead  would  be  very  costly.  It  is  unlikely  that 
manaaenent  woula  support  an  exoeri ^ent  of  this  type. 

- Competition  among  programmers  would  probably  not  allow 
accurate  results  to  be  obtained. 

- Programmers  may  feel  that  the  error  data  would  be 
used  for  performance  evaluation. 

In  addition  to  Gathering  Quantitative  error  data  an 
attempt  was  made  to  obtain  other  information  acout  the  na- 
ture of  errors  such  as  "why  was  the  error  made?"  or  "how  was 
the  error  discovered?".  This  information  could  help  to  dev- 
ise methods  for  avoiding  certain  types  of  errors.  For  each 
subroutine  a directed  araoh  representation  was  analyzed  with 
respect  to  the  following  core  I ex i tv  measures! 

- Number  of  Nodes 

- Number  of  Arcs 

- Number  of  Statements 

- Number  of  Paths 

- P e a c h a n i 1 i t y of  ''odes 

- Cyclomatic  Number  [3?) 

i. 

The  results  of  this  analysis  are  shown  together  with  the 
directed  graph  representation  of  the  oroqrams  in  the  appen- 
dices. For  some  structures  these  measures  are  not  shown 
because  the  number  cf  oaths  and  the  reachability  inrjex  was 
too  large  to  calculate. 


37 


C.  PROGRAMMING  ENVIRONMENT 

The  underlying  o r c g r a m specifications  of  all  orcjects 
were  well  defined  ana  not  known  by  the  experiment  programmer 
before  the  experiment.  The  experiment  programmer  was  fami- 
liar with  the  objectives  of  the  experiment  and  willinq  to 
record  accurate  data  to  the  nest  of  his  knowl edqei  even  if 
recorded  facts  seemed  to  be  unfavoraole  for  himself. 

Throughout  the  experiment  software  development  was  done 
in  a structured  and  phase  oriented  approach  using  structured 
programming  and  known  software  development  techniques,  such 
as  top-down  design,  modularization,  decision  tables,  etc.. 

As  a programming  environment  the  0S/3b0  system  was 
chosen  as  one  o * the  most  representative  batch  processing 
systems  available.  batch  processing  was  always  usea  except 
for  the  final  test  ang  debug  runs  for  project  * 0,  because 
this  program  was  oesionea  to  run  in  a time  sharing  environ- 
ment. During  the  experiment  a'l  en v i ronmen t a 1 factors,  such 
as  operating  system  and  eatiicment  being  used  and  the  pro- 
gramming 1 anauaoe  remained  unchanged.  factors  related  to 
the  experiment  programmer,  such  as  ability,  knowledae,  and 
experience  did  not  vary  considerably  d 
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urina  the  exceri-ent 
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QUANTITATIVE  results 


A.  DISTRIBUTION  OF  PROGRAMMING  EFFORT 


In  Table  1 the  distribution  of  project  time  is  shown 
w i t n respect  to  the  major  software  development  chases. 
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TABLE  1 

DISTRIBUTION  OF  PROJECT  TIME  DURING  T*£  EXPERIMENT 


H.  ORIGIN  OF  ERRORS  VS  DETECT  ION  OF  EDROPS 


Tne  final  statistics  of  each  project  contains  tables 


showing  the  numbers  of  errors  made  ana  the  number  of  errors 
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L 


found 


with  rescec  t 


to  each  of 


the  software  development 


phases. 


A summary  of  tnese  tables 


is  presented 


in  Table  2 . 


tt  OF  ERRORS  ROUND 


PROJECT 

PHASE 


Project 
a 1 


Project 
a 2 


Project 
a 3 


Project 
s a 


TOTAL  (Percentage) 


DESIGN  ! 

2 

- 4 - • 

1 

1 

1 

1 

1 

1 

1 

- 4 - 

1 

• 

3 

( 1.7 

X) 

CODING  : 

22 

1 

1 

2 1 

1 

1 

1 ! 

a 

1 

1 

4 6 

(27.7 

X) 

DEBUGGING  1 

IQ 

1 

1 

53 

1 

• 

3 ! 

45 

• 

1 

120 

(oO . o 

X) 

TESTING  ! 

1 

1 

1 

- 4 -• 

1 

1 

t 

1 

1 

1 

• 

• 4 - 

2 

( 1.2 

X) 

SUM 


4U 


7 5 


4 


50 


1 7 3 ( 1 0 0 */. ) 


OF 


FRRORS  made 


PROJECT 
Pm  ASE 

• 

1 

• 

» 

Project  ! P 
e 1 J 

r o j ec  f 
a 2 

1 P r o j ec  t 
! a 3 

! ° 

i 

i 

reject 
a 4 

1 

1 

1 

1 

total 

(Percent  aae ) 

DESIGN 

l 

1 

5 ! 

2 0 

1 

1 

i 

• 

10 

1 

f 

35 

(20.2  X) 

CODING 

1 

1 

38  ! 

53 

! 4 

i 

i 

37 

1 

I 

1 32 

( 7 o . 3 X) 

DEBUGblN 

g; 

i ; 

2 

1 

1 

• 

i 

3 

1 

• 

o 

k-T 

TESTING 

i 

i 

i 

• 

1 

1 

i 

i 

1 

• 

0 

o 

• 

o 

SUM 

< 

i 

44  ; 

75 

! 4 

i 

i 

50 

1 

1 

173 

(100  X ) 

table  ? 

ERROR  DETECTION  VS  ERROR  OR  I G 1 1 
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C.  ERROR  CORRECTION  TI^E  AND  TIve  BETWEEN  ERRORS 

1.  Error  Correction  Time 

The  mean  time  to  correct  an  error  calculatec  over 
alt  173  errors  was  \2.2  man  minutes. 

2 . Time  Between  Error  Detections 

The  mean  time  ppt«“en  error  detections  calculated 
over  all  projects  was  61. o man  minutes. 

D.  ERFl°  TYPES 

All  errors  which  occurred  during  the  exoeriment  could  be 
identified  by  one  of  the  creviously  defined  error  types.  An 
overview  of  error  types  with  respect  to  their  frequency  of 


occurrence 

is  cresented 

in  Table 

3. 

E r r o r 

tyoes 

which  did 

not  occur 

are  not  1 i 

s t ed . 

I n 

add i t i on 

to  the 

informat  ion 

given  in  Taole  3 a bar 

chart 

(Table 

A ) IS 

Shown  w 

h i c h i den- 

t i f i es  the 

most  C ommon 

error 

types. 

ERROR 

1 

1 

number 

OF  ERRORS 

! total 

TYPE 

1 P ro  j ec  t 

{Project 

! P r o j ec  t 

{Project 

f 

1 

: * i 

! « 2 

! “ 3 

! a a 

1 

1 

4- - - - 

t ------- 

4- 

4- 

4--- 

D 3 

! 3 

! 3 

• 

i 

! 2 

! 8 

D 7 

1 

1 

1 

1 

i 

• 

: i 

: i 

DP 

: i 

1 10 

i 

i 

» 

i 

: 1 1 

DIO 

i 

i 

i 1 

i 

i 

* 

i 

: i 

01  1 

: i 

1 3 

i 

i 

i 

i 

! 4 

D12 

i 

i 

! 2 

i 

t 

! S 

: 7 

D 1 3 

i 

« 

1 

l 

i 

i 

: i 

i i 

D 1 5 

i 

i 

! 1 

i 

i 

: i 

: 2 

4-  — - — - - 

4-  - ------ 

+ ------- 

4-- ------ 

4>  ---  - ------ 

Cl 

! 1 

! 1 

i 

i 

i 

i 

! 2 

ca 

: i 

1 

i 

i 

i 

i 

: i 

C 5 

! 1 

1 

i 

i 

* 

i 

» 

! 1 

C 6 

: i 

• 

1 

l i 

! 2 

! 4 

C 7 

i 

i 

1 

1 

i 

t 

; l 

1 1 

C 8 

! 2 

! 1 

i 

i 

t 

i 

: 3 

CP 

1 

• 

: 2 

i 

i 

! 1 

: 3 

CIO 

: i 

! 2 

* 

i 

t 

1 

: 3 

C 1 1 

: 3 

! 2 

• \ 
t 

1 

1 

: s 

C12 

! 4 

! 2 

i 

i 

i 1 

! 7 

CIS 

i 1 

1 

1 

i 

i 

• 

1 

i 1 

C 16 

1 

• 

! 1 

i 

i 

1 

• 

i 1 

C 1 7 

1 1 

! 5 

i 

i 

: 3 

! 9 

C IP 

: i 

1 

1 

: i 

i 

• 

! 2 

C20 

i 2 

: i 

i 

i 

i 

» 

! 3 

C21 

• 

» 

: 2 

: i 

: 2 

! 5 

C23 

1 6 

1 s 

• 

• 

; i 

1 12 

C?4 

1 2 

: i 

: i 

i 

i 

! 4 

C2o 

• 

l 

! 1 

i 

i 

i 

i 

! 1 

C 2 7 

: 2 

! 3 

i 

• 

! 9 

! p 

C28 

! 5 

! 6 

i 

i 

! 6 

! 1 7 

C 29 

1 

1 

: 2 

• 

i 

1 

1 

1 2 

C30 

1 

t 

: i 

i 

i 

• 

1 

! 1 

-------- 

4 - - - - - — 

-------- 

4 — — — — — — — 

4-  - - --  --  --  -- 

A 1 

: 2 

! lu 

i 

i 

: to 

! 32 

A2 

: 2 

; i 

i 

i 

: i 

! 4 

A3 

i 

i 

i 

i 

i 

i 

: i 

: i 

4 - - - 

4-  - ------ 

———————— 

4 — — 

B 3 

: i 

i 

i 

• 

i 

i 

t 

: i 

Ba 

i 

i 

1 2 

• 

i 

i i 

! 3 

TABLE  5 


ERROR  DISCOVERY  ,-JlTh  RESPECT  TO  E'RROR  Types 


ERROR 

TYPE 


FREQUENCY  OF  OCCURRENCE 


♦ 


D 3 

I ******** 

09 

J *********** 

01  1 

J * * * * 

012 

J ******* 

C b 

J * * * * 

Cl  1 

J ***** 

C 12 

J ******* 

C 1 7 

J ********* 

C21 

I ***** 

C 2 3 

{ ************ 

C2a 

I * * * * 

C27 

J ********* 

C28 

J ***************** 

A 1 

I******************************** 

A2 

1 * * * * 

Other 

* 

I 

Errors 

J *********************************** 

♦ 


TABLE  a 

MOST  FREQUENT  ERROR  TYPES 


E.  ERROR  OCCURRENCE  REL A ( ED  TO  COMPLEXITY  MEASURES 


One  of  toe  objectives  of  the  exoeriment  * a s to  determine 


the  re 

1 a t i 

onshicii  i 

f any. 

between  error  occurrence 

ana  the 

s t ructu 

r a 1 

orooert ies 

of  subroutines.  Of 

oa r t i c u 

1 a r i n - 

t erest 

was 

the  ana 

lysis 

of  structural 

croc? r t i 

e s with 

resoec  t 

t 0 

the  error 

simulation  work  done 

by  Schne i dew i nd» 

Howard 

and 

Ki rchoaessner 

(331.  In  a d a i t i 

on  the  eye  1 oma  t i c 

nymDe  r 

as  de  f i neo  by 

RcCabe 

(32)  for  soft 

ware  eng 

i nee  r i nq 

cu roose s i 

was  used 

i n t 

no  analysis.  The  results 

0 f this 

u3  - 


analysis  are  shewn  in  lah I » 8 


The  f O 1 I O w i n a definitions 


and  terms  are  used  in  Table  S 


No  Number  of  Paths  (minimum  number  of  paths:  no 

Iood  traversed  more  than  once  in  succession) 

V Cvctomatic  Number:  number  of  independent  circuits  = 

number  of  arcs  “ number  of  nodes  + ?. 

R Reachability:  summation,  over  the  nodes,  of  number 

of  >*ays  of  reaching  a node 

r Averaae  reachability:  R/numoer  of  nodes 

S Number  of  Source  Statements 

e Number  of  Errors  Pound  in  Actual  Rroaram 

Tf  Labor  Time  Required  to  Find  Errors 
(Since  Previous  Error  betection) 

Tc  Lacor  Time  Required  to  Correct  Error 


All  of  the  above  are  with  respect  to  a single  crcqram. 
The  definition  of  V includes  an  i me  1 i c i t arc  connecting  the 
start  and  terminal  noces  (stronaly  connected  gcach)  , 
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f 


A 


table  5 

C0MPLEX I TV  MEASURES  VS.  ERROR  PROPERTIES 
Part  A - Procedures  V.ith  One  or  vore  Errors 

COMPLEXITY  measures  error  properties 


Project/ 

Procedure 

Nic 

V 

R 

r 

S 

e 

(Nan- 

mins) 

Etc 

(Man- 
mi  n s 

- - 

- 

- 

- 

■ 

- 

- - - - 

1 / 1 

2 

2 

7 

1 .4 

1 4 

1 

35 

10 

1 /s 

* 

0 

* 

* 

2b 

5 

95 

53 

1 / o 

# 

5 

* 

* 

7 

2 

37 

35 

1/8 

* 

5 

* 

* 

21 

1 

35 

15 

1 /9 

2 

2 

8 

1.333 

6 

1 

1 15 

20 

2/1  . 1° 

1 

1 

2 

1 .0 

3 

1 

10 

1 0 

2/  1 .23 

1 

1 

2 

1 .0 

1 1 

1 

1 1 0 

10 

2/2.2 

2 

1 

4 

1 .0 

8 

1 

15 

10 

2/7 

3 

2 

7 

1 . 4 

15 

3 

230 

45 

2/9 

72 

8 

370 

1 ° . 4 74 

45 

3 

140 

185 

2/10 

9 

4 

25 

2.778 

1 8 

1 

10 

5 

2/1  1 

* 

<5 

* 

* 

54 

3 

950 

o5 

2/12 

5 

2 

1 3 

1 .444 

34 

2 

300 

30 

2/15 

* 

4 

* 

* 

1 9 

1 

5 

1 

2 / 1 o 

* 

5 

* 

* 

30 

2 

150 

20 

2/1  0 

12 

4 

50 

2.941 

2b 

t 

5 

15 

2/21 

* 

1 b 

* 

* 

04 

8 

750 

145 

3/3 

2 

2 

b 

1.5 

1 3 

1 

b0 

5 

- as  - 


* verv  large  value 


table  s 

(coot inuefl) 


Part  A 

- Procedures 

m i t h One 

o r 

vo  r e 

Errors 

Project/ 

P roceoure 

i'  Jo 

V 

R 

r 

S 

e 

Df 

(Man- 

mins) 

Etc 

(Man- 

mins 

- - 

- 

- 

- 

— 

• 

* • • • * 

4/7 

4 0 

6 

151 

3.438 

83 

1 

120 

5 

«/ 1 5 

1 6 

5 

O 4 

4.267 

28 

1 

2 0 

15 

4/14 

* 

e 

* 

37 

5 

255 

o5 

4/1  5 

4 

3 

1 2 

2.0 

1 3 

2 

40 

35 

4/21 . 3 

7 

3 

34 

4.657 

lo 

1 

0 

5 

4/22 

* 

7 

* 

* 

34 

1 

1 o 0 

30 

4/23 

1 8 

5 

o 0 

4.615 

24 

t 

125 

10 

4/27 

5 

4 

2 3 

2.001 

1 8 

3 

3b  0 

120 

4/28 

* 

5 

* 

* 

35 

2 

90 

50 

4/29 

321 

1 3 

1 4bB 

54. 370 

49 

5 

1 125 

164 

4/30 

6 

4 

24 

2.4 

1 9 

1 

60 

30 

4/31 

* 

4 

* 

* 

27 

1 

30 

1 0 

4/33 

1 4 

4 

7b 

7.o 

1 7 

2 

1 35 

10 

* Very  lane  value 


4b 


table  s 

(continuea) 


Part  B - CoTDlexitv  Measures  Tor  Procedures 
with  Zero  Errors 

Project/ 

Procedure  No  V W r S 


1/? 

3 

2 

8 

1.333 

6 

1/3.1 

1 

1 

2 

1.0 

8 

1/3.2 

1 

1 

2 

1 .0 

1 1 

1/3.3 

1 

1 

2 

1 .0 

4 

l/a 

o 

3 

2o 

4.333 

1 8 

1 / 7 

7 

3 

40 

5.0 

15 

2/1.1 

1 

1 

2 

1 .0 

3 

2/1.2 

1 

1 

2 

1 .0 

3 

2/1.3 

1 

1 

2 

1 . 0 

3 

2/  1 . a 

1 

1 

2 

1 .0 

3 

2/1  .5 

1 

1 

2 

1 .0 

3 

2/1.6 

1 

1 

2 

1 .0 

3 

2/1.7 

1 

1 

2 

1 .0 

3 

2/1  .8 

1 

1 

2 

1 .0 

3 

2/1.9 

1 

1 

2 

1 .0 

5 

2/1.10 

1 

1 

2 

1 .0 

5 

2/1.11 

1 

1 

2 

1 .0 

5 

2/1  . 12 

1 

1 

2 

1 .0 

1 3 

2/1.13 

1 

1 

2 

1 .0 

3 

2/1  . 1U 

1 

1 

2 

1 .0 

3 

2/1  . IB 

1 

1 

2 

1 .0 

3 

2/1.16 

1 

1 

2 

1 .0 

3 

2/1.17 

1 

1 

2 

1 .0 

3 

2/1.18 

1 

1 

2 

1 .0 

3 

2/ 1 .20 

1 

1 

2 

1 .0 

3 

a/1.1 

1 

1 

2 

1 .0 

2 

4/1.2 

1 

1 

2 

1 .0 

2 

4/1.3 

1 

1 

2 

1.0 

7 

4/1.4 

1 

1 

2 

1 .0 

5 

4/1.5 

1 

1 

2 

1 .0 

7 

4/1.0 

1 

1 

2 

1 .0 

5 

4/1.7 

1 

1 

2 

1.0 

5 

4/1.8 

1 

1 

2 

1 . 0 

5 

4/1  .9 

1 

t 

2 

1 .0 

5 

4/1.10 

1 

1 

2 

1 .0 

4 

4/1.11 

1 

t 

2 

1.0 

3 

4/1.12 

l 

1 

2 

1 .0 

3 

4/1.13 

1 

1 

2 

1 .0 

3 

4/1.14 

1 

1 

2 

1 .0 

3 

4/1.15 

1 

1 

2 

1 .0 

3 

4/1.10 

1 

1 

2 

1 .0 

3 

4/1.17 

1 

1 

2 

1 .9 

3 
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TABLE  5 
(con  r i nusa ) 


Part  6 - Comclexity  Measures  for  Proceoures 
with  Zero  Errors 


P ro  j ec  t / 


Procedure 

flc 

V 

R 

r 

S 

4/1.18 

1 

1 

2 

1 .0 

3 

4/1  . IP 

I 

1 

2 

1 .0 

5 

4/1.20 

l 

1 

2 

1.0 

5 

4/1.21 

1 

1 

2 

1 .0 

0 

4/1 .22 

1 

1 

2 

1.0 

9 

4/1 .23 

1 

1 

2 

1 . 0 

b 

4/2.1 

2 

1 

u 

1 .0 

8 

4/2.2 

2 

1 

4 

1 .0 

9 

4/2.3 

2 

1 

4 

1 .0 

9 

4/3 

2 

2 

0 

1 .5 

4 

4/4.  1 

2 

2 

8 

1 .0 

7 

4/4.2 

2 

2 

8 

1 .0 

0 

4/5 

8 

4 

38 

4.222 

5c 

4/b 

4 

1 

0 

1 .0 

24 

4/2.  1 

2 

2 

8 

1 .0 

1 3 

4/8.2 

2 

2 

8 

1 .0 

1 3 

4/8.3 

2 

2 

8 

1 .0 

1 0 

4/8.4 

2 

2 

8. 

1 . o 

9 

4/8.5 

2 

2 

8 

1 .0 

12 

4/4 

1 0 

5 

45 

7.417 

21 

4/10 

12 

5 

65 

4.043 

49 

4/11 

7 

3 

38 

5.429 

19 

4/12 

* 

4 

* 

* 

20 

4/lb.  1 

2 

2 

6 

1.5 

o 

4/10.2 

2 

2 

0 

1 .5 

1 2 

4/10.3 

2 

2 

0 

1 .8 

9 

4/10.4 

2 

2 

6 

1.5 

1 0 

4/17 

lo 

1 

18 

l .0 

21 

4/  1 8 

8 

4 

42 

3.818 

21 

4/14 

4 

3 

lo 

2 . oo 7 

1 1 

4/20 

3 

2 

9 

1.125 

1 3 

4/21.1 

7 

3 

34 

4.857 

1 4 

4/24 

lo 

7 

83 

4 . 3o8 

1 9 

4/28.1 

2 

2 

8 

1.333 

15 

4/25.2 

2 

2 

8 

1 .333 

1 0 

4/25.3 

2 

2 

8 

1 .333 

1 7 

4 / 2 0 

3 

3 

15 

1 .5 

19 

4/32 

7 

3 

30 

5.143 

15 

4/34 

2 

2 

5 

1 .25 

1 5 

* Verv  I arpe  val u® 
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Using  the  results  of  Part  A in  Table  5 , the  following 
correlation  coefficients  were  calculated: 


V 

vs. 

e : 

. 7834 

s 

vs  . 

e : 

.5880 

V 

V s • 

2 T f 

: . b 7 5 4 

V 

V s • 

2Tc 

: . 7229 

s 

vs. 

ZTf 

: .5902 

s 

vs. 

ITc 

.509  1 

V 

V s . 

S : 

. 790  3 

The  relationship 

between  rear  complexity 

error  occurrenc 

e i s 

shown  in  Table  b . 

Zero  Errors  One  Or  Pore  Errors 


a 1 

P roc  eou  res 

3 1 Procedures 

P'ean  Cvclomatic 
Number  (VJ 

1 .bR 

4.79 

M e a n Number  of  Source 
S t a t emen  t s (SI 

9.33 

27.23 

TABLE  o 

RELATIONSHIP  BETWEEN  ME  AN 

complexity 

VALLES  A '10  ERRORS  FOUND 


- 49  - 


VI 


SUMMARY  AND  CONCLUSIONS 


A.  general  remarks 

Due  to  limited  man  power  and  time  constraints  the  amount 
of  test  data  was  too  small  to  allow  major  conclusions. 
Furthermore  the  scope  of  the  experiment  did  not  include  some 
important  oroqramminq  problems  such  as  real  time  and 
matnematical  problems.  However,  takinq  all  constraints  into 
consideration  some  qualitative  aspects  were  recognized  which 
could  be  important  factors  in  subsequent  studies. 


9.  SIGNIFICANCE  OF  ERROR  TYPES 


A 1 t hour" 

the  c a t ego r i z a t i on 

ang 

de  f i n i t 

ion  of  error 

tyoes  was  no' 

considered  complete. 

a 1 1 

e r r o r s 

detected  in 

the  experiment 

could  be  identified 

u s i 

ng  these 

error  oe f i r i - 

tions.  The  most  frequent  error  type  " a l " (^gnual  Error) 
seems  to  ce  relatec  to  the  nunher  of  source  statements 
whereas  the  frequencies  of  the  remaining  errors  seem  to  oe 
related  to  the  complexity  of  the  oroqramminq  problems. 

The  recording  of  errors  with  resoect  to  their  tyoes  con- 
tributes to  a learning  process  which  enables  the  programmer 
to  reduce  the  number  of  errors  made  on  suoseouent  projects. 
It  was  felt  that  this  concect  could  also  be  effectively  usea 
in  a commercial  software  production  environment.  The  error 
recording  would  have  t o be  private  to  tne  individual 
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programmer  ana  usei  by  the  programmer  alone 


Otherwise  it 


would  have  a negative  influence  on  working  atmosphere  and 
performance.  If  the  evaluation  process  were  assisted  by 
appropriate  software  oac*aaes»  the  overhead  in  analyzing 
errors  would  be  reduced  to  a great  extent.  The  overheaa  of 
recording  the  errors  for  the  individual  programmer  tends  to 
decrease  after  the  recording  svstem  has  been  learned. 

C.  complexity  MEASURES 

Several  measures  of  complexity  calculated  for  each 
subroutine  are  presented  in  the  aopenaencis  tcaether  with 
suoroutine  directed  graph  representation.  nhen  the  number 
of  errors  found  in  procedures  was  correlated  with  cyclorat* 
ic  number  ana  number  of  source  statements.  the  correlation 
coefficients  were  hioher  than  for  other  complexity  measures 
( 3 4 J . It  also  appeared  that  these  two  measures  were  related 
to  the  total  error  detection  and  total  error  correction 
times.  It  was  learned  that  tryina  to  keen  the  cvclomatic 
numoer  small  not  only  reduced  the  numcer  of  errors  out  also 
contributed  to  the  reduction  of  aebugainq  and  testing  ef- 
fort. Tris  experience  supports  the  results  of  McCabe's 
analysis  of  structural  nrocert ies  of  programs  with  respect 
to  difficulties  in  testing  and  debugging  1321.  It  should  be 
noted  that  the  influence  of  other  factors  such  as  usage  of 
structured  oroaramming  techniaues  and  working  habits  of  the 
exnerimenr  programmer  also  contributed  to  these  results. 
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SOFTWARE  ENGINEERING  ASPECTS 


Throughout  the  experiment  comments  «ere  made  to  indicate 
oossiole  clues  about  error  sources  or  ways  to  avoid  certain 
types  of  errors.  An  evaluation  of  these  comments  showed 
that  some  of  these  errors  would  have  been  avoided  by  the 
usaqe  of  decision  tables  or  aoolying  a proper  desk  test. 
Many  key  Punch  errors  of  tyne  "A]"  were  not  recognized  dur- 
ing the  process  of  ounchina  caros , because  the  oesign  of  the 
key  punch  machines  (ISM  po ) does  not  provide  an  immediate 
control  of  characters  beinq  ounched.  It  is  desirable  to  see 
the  result  of  each  key  stroke  instantaneously.  The  freauen- 
cy  of  error  occurrence  of  tyre  "Manual  Error”  in  former  pro- 
jects done  by  the  same  programmer  using  console  editing  was 
considerably  lower  than  the  error  frequency  discovered  in 
this  experiment. 

Software  engineerino  concents  such  as  top-down  design, 
design  review  and  Structured  walk  Throuah  were  used 
throughout  all  projects.  These  methods  had  a major  influ- 
ence on  the  software  development  process  and  were  considered 
to  be  extremely  useful. 

The  usage  of  ALGOL  ft  had  a major  influence  on  the  pro- 
gramming style.  Variable  names  were  chosen  to  make  programs 
se 1 f -doc umen t i no  to  a areat  extent.  Therefore  most  proaram 
parts  were  easy  to  read  and  to  understand  which  implied  a 
reduction  of  the  deouccing  effort. 
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Another  conc?Dt  which  was  successfully  e m o 1 c v e a was  the 
idea  of  using  identifiers  which  differ  by  two  or  more  char- 
acters. The  idea  of  imolementing  some  kind  of  " H a m m i n a Dis- 
tance" [35]  between  identifiers  was  derived  from  bitter 
experience  in  former  projects  and  helced  to  avoid  errors 
which  occur  when  a single  error  of  tyoe  "Manual  Error"  or 
"Cental  Error"  produces  a different  identifier  which  matches 
with  one  of  the  previously  defined  identifiers.  Thus  this 
kind  of  error  did  not  occur  during  the  compiling  process. 

e.  conclusions 

Errors  in  software  projects  appear  to  be  related  to  pro- 
gram structure  and  tc  the  number  of  source  statements. 
Although  this  experiment  was  not  larae  enough  to  state  these 
relations  in  a formal  manner,  the  results  »ere  encouraging. 
Cyclomat  ic  numoer  and  number  of  source  statements  could 
prove  useful  as  guidelines  for  managing  proaramming  pro- 
jects. 

Th“  usage  of  foo-down  desion,  structured  orogramming 
technidues  and  other  software  development  tools  should  be 
mandatory  for  orogramming  projects.  The  recording  and 
evaluation  of  software  errors  would  be  a reasonable  accroach 
to  avoid  errors  in  programming  projects  ana  to  reduce  cost. 

Detailed  oenuddi na  on  a modular  basis  and  development  of 
large  orograms  by  stecwise  inteoration  of  thoroughly  ae- 
b'jnged  modules  seems  to  be  a good  strategy  to  achieve 
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reliable  prograans 


F.  recommendations 

Similar  experiments  would  reveal  mere  of  the  relations 
between  software  errors  and  complexity  of  programs.  More 
definite  information  about  t^e  utility  of  complexity  meas- 
ures could  be  obtained  by  usina  these  measures  on  large- 
scale  software  projects  in  a commercial  production  environ- 
ment . 
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APPENDIX  A 


PROJECT  DESCRIPTION 

Project  « : 1 

Project  title  : PALINDROMES 
Progran’cer  : HOFFMANN 
Programming  Language  : ALGOL 

Programming  environment:  I8M/3bO/67,OS/vVT»eATCH 

Desian  notes  : see  ANNEX  A 

Program  listing  : see  ANNEX  5 

Coding  notes  : see  ANNEX  C 

Debugging  notes  : see  ANNEX  D 

Error  Listing  : see  ANNEX  E 

Final  statistics  : see  ANNEX  F 

Graphical  reoresent at i on  : see  ANNEX  G 

Test  phase  description:  see  ANNEX  H 

Starting  gate:  20  JAN  77  Ending  date:  b FEB  77 


experiment  description 


1.  Project  description: 

FIND  PALINDRO^S 

A palindrome  is  defined  as  a character  string  of  length 
n,  which  has  the  following  characteristics: 

character  [il  = character  In  - i - 11  for  all  i from 

i =0  to  i=n/2  (integer  division)  and  n >=  2 
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Given  an  incut  string  of  length  n,  where  2 <-  n <-  2 5b 


find  all  occurrences  of  oa 1 i naromes  . 

The  input  is  given  in  form  of  ounched  cards. 

The  length  of  the  string  ("1")  preceeas  the  input  of  the 
character  string. 

Ml  information  being  read  should  be  printed. 

Give  as  a result  the  number  of  palindromes  of  each  length 
and  the  Position  of  the  starting  character  of  each  palin- 
drome in  card  number  and  character  position. 

Fina  only  maximal  length  palindromes,  (i.e.  "ABC  CBA"  should 
be  recorded  as  the  palindrome  of  maximal  length,  whereas  the 
included  palino homes  "BC  CB"  and  "C  C"  should  be  exclud- 


2.  Programmer's  background: 

a)  Experience  in  programming: 

Cc  t 1 970  - ’"‘ay  1971  Proaramming  courses 

May  1971  - April  1972  Module  Programmer 

May  1972  - June  1974  iNo  r k in  Test  and  Simulation  Depart- 
ment at  the 

NiAVAl  command  and  control  systems 
command  cfedepal  gepman  navy) 

Testina  of  tactical  real  time  systems 
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March  1975  - J an  1977  Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL/  Monterey,  Computer  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  and  simulation. 

c)  Experience  in  the  area  of  the  given  problem:  None. 

d)  Experience  in  the  proqamminq  1 anguaqe  being  used: 
Experience  over  a period  of  18  months  in  more  than  10 

programming  orojects.  (Total  number  of  source  statements 
produced  durina  that  time  was  more  4000.) 

3.  Psychological  factors: 

a)  Did  the  programmer  like  the  project?  Yes. 

0)  How  does  the  programmer  like  the  programming 
1 anguage? 

Favorite  programming  language. 

c)  was  the  programmer  satisfied  oy  tne  way  the  o r o b I e m 
was  specified? 

Only  minor  criticism. 

d)  How  did  the  programmer  like  the  programming  environ- 
ment ? 

The  facilities  (study  room,  card  punch  room)  were  not 
conducive  to  efficient  programming  because  of  restricted 
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baa  lighting  and  noise, 
e)  Other  factors: 

The  recording  of  the  experiment's  cata  during  the 
ject  affected  speed  and  concentration  considerably. 
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4 . Comments  on  Documentation 

For  the  documentation  of  each  software  aevelocment  phase  a 
special  doc umen t a t i on  form  has  been  developed.  These  forms 
are  desiqned  to  provide  a firo  Quideline  for  the  experiment 
programmer  to  record  all  data  of  interest  for  suosequent 
errof  analysis, 

- Beain  and  end  of  each  step  was  recorded  with  resoect  to 
day  and  time. 

- Each  error  was  recorded  when  it  is  aisccverea.  The  error 
was  then  identified  by  a unique  error  number  ( 1 /<?/...). 
Furthermore  the  time  of  discovery  and  tre  error  type  (using 
error  types  listed  in  ANNEX  F ) were  recorded. 

- It  appropriate/  c°mments  about  error  discovery/  reason  why 
the  error  was  made/  etc.  were  documented  in  ANNEX  E 2. 

- For  each  error  the  chase  in  which  the  error  was  made.  the 
phase  in  which  the  error  was  discovered  and  the  time  scent 
to  correct  the  error  was  recorded  in  ANNEX  £1. 

- For  eacn  step  in  any  one  of  the  software  development 
phases  the  day/time  of  beain  and  end  was  recorded.  In  addi- 
tion/ the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
experiment  data. 
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WORKSHEET  for  DESIGN  phase  ANO  DESIGN  REVIEW  phase  of  PROJECT  » 1 


STEP 

« 

problem  and  planned  solution 

! 

! ALTERNATE 
i SOLUTIONS 

1 MAN 
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ERROR  1 COGENT 
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blocks: 
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1 
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f 

• high  are  overlapping: 

J palinaroneSa 

• 

A3BA«A  .ill  ee  recorded  as 

! this  allows  two 

1 

• 
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! solutions: 

1 
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lie  A 1 ways  record 

1 

• 
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1 

1 
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« 

1 

-The  necessary  incuts 

! 2 . Always  record 

• 

1.  length  0*  String 
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« 
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I 
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WORKSHEET  FOR  DESIGN  PHASE  AMO  DESIGN  REVIEW  PHASE  OF  PROJECT  a 1 


! «AN  ! 

STEP 

PROBLEM  AND  PLANNED  SOLUTION 
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ERROR 

COMMENT 
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SOLUTIONS 
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WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  a 1 
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WORKSHEET  FOR  DESIGN  PHASE  AMD  OESIGN  REvIErt  PHASE  OF  PROJECT  a 1 
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annex  c 


Pace  1 of  1 


WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  a : 1 
Beginning  of  Coding  (day/time)  ! 1/28/2200 

End  of  Coding  (day/time)  ! 1/31/1220 

Man  hours  : 7.0  (including  Dunching  of  cards) 


1) 


CODING  ! PROGRAM  PAST 

BEGIN  ! END  ! 

OAX/TImE  CAY/TIME  ! 

ERROR * OA  Y 
a {TIME 

i 

COMMENT 

(incl.  coded  error  tyoes) 

t 1 

1 1 

1)  Record  when  error  is 

1 • 

1 • 

aetectea. 

1/28/2200  1 !-0ata  Definition 

i [-Initialization 

1 t 

3 

2205 

C 2 3 

• < 

* • 

4 

2210 

C23 

11/28/2300  ! 

1/30/1335  [ [-Utilities 

! [(texts. I/O 

5 

1 340 

CIO 

[ ! sucroutioes) 

b 

14C0 

C 8 

1 1 

7 

U05 

Co 

1 1 

8 

1430 

C23 

l 1 

q 

1500 

C23 

1 i 

• i 

10 

1510 

D3 

[1/30/1530  ! 

• 1 

1/30/1O15  1 [-Salingrome 

! ! cnec* 

1 1 

1630 

Cl  Mi sunderstanging 

i i 

i • 

of  design 

[1/30/1700  ; 

• 1 

1/30/1700  ! ! Coding  Review 

12 

1710 

Cl  1 

• • 

13 

1710 

Cll  (wronq  indentation) 

1 1 

14 

1 720 

C 1 7 

1 • 

1 1 

15 

1 725 

C27 

[1/30/1730  [ 

1 • 

1/31/0R30  ! ! °unening  Cards 

1 0 

1030 

C 1 5 

• 1 

17 

1050 

C 4 

18 

1110 

C20 

1R 

1115 

C28 

• 1 

20 

1125 

C 1 2 

21 

1 125 

C1R 

22 

1 1 30 

C24 

1 • 

23 

1135 

C 1 2 

1 < 

1 • 

2« 

1 1 40 

C2u 

[1/31/1220  ; 

Rena  r x s : 

Man  hours  scent  for  ounchinq  caros:  2.8 
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ANNEX  0 


Page  1 o f a 


[ 


WORKSHEET  fop  DEBUGGING  phase 


PROJECT  a : 1 


DEBUG  Run  a : 1 


Begin  of  Debug  Run  (oay/tine)  : 
End  of  Debug  Run  (oay/tine)  : 
a of  Debug  Steos  incl.  in  Deoua 
CPU  tine  for  necessary  conoiles 
a)  l.&R  b)  l.J°  e)  2.13 


1/31/133R 

2/ 04/  1 20  0 

Run:  3 CPU  tine  for  Cebua  run  (sec): 

( sec  ) : 5 . 3 1 

a)  e)  f ) g) 


.02 


Han  lours  for  t n l s Debug  Run  : ,R  (excluoinq  overneao) 

«AN  1) 


STEP ! PROGPAH 
a 1 PART 
1 
1 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL  1 Day | HOURS ! ERROR  1 

RESULT  ! T I ME i /STEP i a | 

II  • 1 

II  1 1 

I 

1 

1 

• 

• 

1 ! All 

Get 

error* ro® 

c o*o i 1 e 

• 1 

1 1 

• 1 

l I 

1 1 

• I 

8 cofl»ol  1 el  1/31  j .3 

! oarts 

errors  113351 

25 

• 

1 1 

• 1 

2b 

1 1 

• 1 

27 

i 

» 1 

• 1 

28 

• 1 

1 1 

2R 

• 

1 I 

1 1 

30 

• 

I » 

• I 

31 

1 • 

1 i 

32 

• 

11/31! 

• 

1 1400! 
1 1 

2 1 All 

Get 

errorfree 

c ono i 1 e 

2 c ono  i l e 1 1 / 3 1 1 .2 

33 

J oarts 

errors  llFOb! 

34 

• 

11/311 

• 

i 

116301 

3 : a ii 

Get 

e r ro  r * ree 

c 0*0 i J e 

0.x.  i 2/ 0 u ; ,u 

J oarts 

111211 

35 

12/04  1 

i 

i 

• 

i 

11  200  1 
• 1 

1 1 

CCHHE'lTS 
AND  CODED 
EPROR  TYPES 


1)  Recorg  u nen 
error 
occurs 


A 1 

a 

C 1 2 
42 
C23 
C 1 2 
Cl  1 
A 1 


C23 

A2 


C 2 A 
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ANNEX  0 


Page  2 of  A 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  » : 1 DEBUG  Run  a : 2 

Beqin  of  Debua  Run  faay/tinf ) : 2/05/  1 350 

End  of  Debug  Run  (day/tine)  : 2/05/1500 

a of  Debug  Steos  incl.  in  Debug  Run:  b CPU  t i«e  for  Debug  run  (sec):  .05 

CPU  time  for  necessary  comoiles  (sec)  : 4.9R 
a)  2.11  b)  l.Rb  c)  1.42  c)  e)  f)  q) 


Han  Hours  (or  tKis  Debua  Pun  : 1.7  (excluding  overhead) 

HAN  1) 


S T£R 

°OOGPAM 

objective  and  expected 

actual 

Day 

HOURS ! EPBOO 

CCWME‘|  T 3 

3 

PART 

RE  'JUT 

RESULT 

tine 

/STEP 

3 

AND  CODED 
ERROR  T»=ES 

1)  Pecnrq  « n • n 

error 

OC  C J r s 

1 

1 J t i 1 i • 

Test  all  Texts  exect 

Pun 

2/05 

.6 

l O 

01  1 

ties 

TEXTS  (all  texts  srould 

error 

13J5 

Ce  Crimea  as  jesiane-j  ) 

2/05 
1 420 

2 

Ut  l 1 i * 

Peoeat  steo  1 

°un 

2/05 

. 5 

37 

C 6 

ties 

error 

1420 

36 

C 2 7 

3G 

C20 

2/05 

1445 

3 

Ut  i 1 i - 

Reoeat  steo  1 

Soac i nq 

2/05 

. b 

t i es 

orob 1 ems 

1445 

40 

C2S 

a 

Ut i 1 i - 

Test  soacing  and  orinfina 

ties 

of  elan*  lines 

(all  formats  snould  ao- 

oear  as  designed) 

O.K  . 

5 

Ut  i li  - 

Test  calculation  of  ngm- 

t i es 

oer  of  incut  cans  for 
minmu1"  length  of  inout 
strinq  (2  characters) 
(croqran  should  read  on- 

O.K. 

ly  one  card) 

b 

Ut  i 1 i * 

Test  detection  and  writ- 

ties 

ing  of  a oal  mdror'e  of 
minimum  length 
(oal  inaror'e  should  oe  de- 
tected ana  listed  as  de- 
signed) 

O.K. 

2/05 

1500 
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ANNEX  0 


Page  3 of  P 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  a : 1 DEBUG  Run  a : 3 

Begin  of  Deoug  Run  (gay/tine)  I 2X05/1500 
End  of  Oebug  Run  (gay/tine)  • 2/ 05/lo00 

# of  Debug  SteDS  incl.  in  Debug  Run:  5 CPU  tine  for  Debug  run  (sec):  8.7R 

CPU  tine  for  necessary  conoiles  (see)  : 3.07 
a)1.51  b)1.5o  c)  o)  e)  ♦)  g) 


fan  hours  for  this  Debua  Pun  : .P  ( exc 1 uai nq  overhead) 

van  1 ) 


STEP 

P ft  OG  ■ " 

OBJECTIVE  AND  EXPECTED 

ACTUAL 

CAY 

ROUPS  1 ERROR 

corven  rs 

8 

P AP’Tt 

RESULT 

RESULT 

t 

/STE  = 

3 

AND  COOED 
ERROR  TYPES 

1)  Pecord  *her 

error 

occurs 

1 

Ut  i 1 i - 

Test  calculations  of  nun- 

2/05 

.5 

t i es 

oer  o*  inout  cans  fcr 
maximum  strina  f e r<  q t * 
(i.e.  2So  characters) 
Ccrogran  s^ou 1 a read  J 

0 

1500 

cards) 

2 

Palin- 

Test  detection  and  -rit- 

dro'n? 

i n g o*  a oali drone  c t 

run 

1500 

X 1 

C 2 9 

Check 

naxinun  lenqth 
(PALI0R0NE  COUNTER  should 
be  1>  palidrone  should  be 
detected  and  orinted  as 
des i gned ) 

error 

2/05 

1530 

• 

3 

Reoeat  steos  1 and  2 

O.K. 

2/05 

, a 

a 

Pal  in- 

Test  detection  and  »rit- 

1530 

orome 

i n g qf  naxinyn  n u n c e r g f 

check 

oalindrones  of  nininum 
1 enqth 

(PALIN0R0ME  counter 
should  be  123i  all  12? 
oalindrones  should  be 
listed  as  designed) 

O.K. 

5 

Pal  in- 

Test  for  String  without 

drone 

cal  indrones 

TEXT3 

1530 

<J2 

33 

Check 

(TEXT3  should  be  orinted) 

not  0 r i n - 

1530 

x 3 

C23 

ted 

2/OS 
160  0 

Renarks:  “run  error"  indicates  ternination  due  to  excess  of  array  limits. 
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ANNEX  0 


Page  R of  4 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  a 5 t DEBUG  Run  * : 4 

Begin  of  Debug  Run  (day/time)  ! 2/05/1600 

End  of  Debug  Run  (aay/time)  ! 2/05/1700 

» of  Debug  Steos  incl.  in  Debuq  Run:  2 CPU  tine  for  Deoug  run  ( 

CPU  tine  for  necessary  conoiles  (sec)  : 1.67 

a)!.o7b)  e)  c)  e)  f)  q) 

fan  hours  for  tnis  Deoug  Run  : .5  (e«cludinq  ovemeao) 

« A M 1 ) 

HOURS  ! EPB0R  1 
TIME 1 /STE=1  = ' 


Initi- 

Test ♦of  inout  o*  illegal 

2/05 

. 5 

al  i z a - 

Strinq  f e n q t ^ 

D.K  . 

1600 

t t on 

(oroqra*  shoulo  orint  a 
*aminq  ana  ter-inate) 

Pal i n- 

Test  ♦or  overlaocina  oa- 

aro^e 

1 i ndrcres 

c^eck 

(sftoula  be  recorded  as 
t «*o  oa  1 i nopg^es  ) 

O.K. 

2/05 

1630 

STEP! 

a 


PROGRAM 

PART 


OBJECTIVE  AND  EXPECTED 
RESULT 


ACTUAL 

result 
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sec ) : .22 


COMMENTS 
A VC  CODED 
ERROR  TfPES 


1 ) Record 
error 
occurs 


ANNEX  El 


°age  1 o f 2 


ERROR  LISTING 


PROJECT  « : l 

Begin  of  Project  (day/time)  : 1/29/10R0 
End  of  Project  (day/tine)  ! 2X06/1620 
Man  hours  for  total  project  : 21.9 


EPBCR 

a 

phase 

in  which 
E°R0R  was 
o i s- 
coverea 

PWASE 
in  which 

ERROR  was 
nade 

ERRQP 

Type 

(see  ANNEX  F ) 

TIRE 

scent  to 
solve  t "e 
EPROR 
(Man 
n i n . ) 

3 of  OTd£R 
STATEMENTS  OR 
PARTS  CF  THE 
PROGRAM 

AFFECTED 

i 

Oes i an 

Oesi an 

D3 

S 

Data  i nou  t on  1 y 

2 

Design 

design 

03 

S 

Data  crintout 

3 

Coding 

Codi ng 

C23 

S 

Indeuing  within 
subsequent  sue* 
rout i ne 

a 

Coa i nq 

Coding 

C23 

3 

5 

Coat  nq 

C o a i n q 

CIO 

2 

b 

Coding 

Ceding 

C 9 

S 

7 

Coot  nq 

Cod i nq 

C 6 

s 

8 

Ceding 

Coding 

C2  3 

s 

q 

Coding 

Codi n g 

C 2 3 

10 

3 s t a t e»ent s 

10 

Coding 

Design 

03 

30 

ahole  sucrou- 
tine  affected 

1 1 

Codi ng 

Codi ng 

Cl 

1 0 

nhole  surrou- 
tine  a * < ec  t ed 

12 

Coding 

Coding 

Cll 

S 

1 J 

Coai ng 

Coding 

Cl  1 

3 

tu 

Coat  nq 

Coding 

C 1 7 

S 

IS 

Coama 

Coding 

C 2 7 

s 

16 

Cod i nq 

Coding 

CIS 

s 

1 7 

Codi nq 

Cod  t na 

C« 

5 

18 

Cod i nq 

Coding 

C 2 0 

S 

IP 

Cod i nq 

Coding 

C22 

5 

20 

Coai ng 

Codi ng 

C 12 

5 

21 

Coding 

Coding 

C 1 6 

S 

22 

Coding 

Cod i na 

C2a 

10 

23 

Coding 

Coding 

C 1 2 

3 

2j 

Coai ng 

Coding 

C2« 

IS 

dhole  suorcu* 
tine  affected 

2S 

Oeougq i nq 

Coding 

A 1 

10 

26 

Debuqd i nq 

Cod  t nq 

C5 

5 

27 

Oebuqq i ng 

Coding 

C 12 

S 

J 
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ANNEX  El 


°age  2 of  2 


PROJECT  a : l 


ERROR  LISTING 


3egin  of  Project  (oay/tine) 
End  of  Project  (oay/time) 
Man  hours  for  total  project 


1/28/1040 

2/00/1620 

21.3 


ERROR 

9 

PHASE 
in  i c n 

ERROR  -as 
dis- 
covered 

PHASE 
in  .hicr 
ERROR  -as 
made 

ERROR 

type 

C see  ANNEX  F 1 

TIME 

scent  to 
solve  the 
ERROR 

(Man 
n i n . ) 

a of  OTHER 
STATEMENTS  or 
parts  of  The 
program 
affected 

28 

Debuqa i nq 

Coni ng 

A 2 

5 

29 

Defcuga i na 

Coding 

C23 

3 

3 0 

Oecuqgi nq 

Coding 

C 1 2 

5 

31 

Debuggi nq 

Cedi ng 

Cl  1 

3 

32 

OeOugq i ng 

Coding 

A 1 

3 

3 3 

OeDuga i nq 

Coding 

C23 

5 

34 

Oecuoqi nq 

Coding 

A 2 

5 

35 

Oebugq  t ng 

Design 

C 2 8 

10 

3b 

Oeouoqi nq 

Design 

01  1 

25 

37 

Oeouqgi nq 

Cod i ng 

C 8 

1 0 

writ i na  o * c a- 
1 i nq  r one  s 

38 

Deouoo i ng 

Coding 

C2  7 

5 

3R 

OeOugq i ng 

Cod i ng 

C20 

10 

flritinq  of  oa- 
1 i nq  r one  s 

40 

Debuggi ng 

Coding 

C23 

20 

Whole  subrou* 
tine  a * f ec  t ed 

41 

Decugg i nq 

Coding 

C23 

15 

42 

Deouga i ng 

Oebugqi ng 

85 

3 

43 

Debugqi nq 

Coo  1 ng 

C29 

15 

uu 

T e s t i n g 

Design 

DR 

20 

76 


ANNEX  E2 


Paqe  1 of  2 


ERROR  LISTING  (COMMENTS) 


ERROR 

0 

day 

TIME 

COMMENTS 

(EVIDENCE,  THOUGHTS,  «HY  «AS  THE  ERROR  MADE? 
WHY  AND  HOW  y»AS  THE  ERROR  DISCOVERED? 

ERROR  SLOCKING,  etc.) 

l 

1/28 

Lack  of  concentration  (steo  left  out,  whict  the  orogranmer 

1505 

was  aware  of ) 

2 

2115 

This  design  steo  was  costooned  ana  forgotten  later  on. 

3 

2205 

Proqrawfl?r  d i a not  c n e c « t h • manual . (Discovered  -hile 

a 

2210 

examining  the  solution  o*  a similar  opcdI in  an  old  program.) 
sane  as  3 

5 

1/30 
I 3«0 

6 

1 u00 

7 

l a05 

3 

1 a 3 0 

error  «*as  found  while  examining  the  list  of  error  tvees 

<5 

1500 

in  connection  w i t h errors  3 and  4. 

Error  was  found  while  reaoinq  the  programming  manual  . 

10 

1510 

Errors  10  and  ll  discovered  durinq  a 3es<  test. 

i i 

1630 

see  10 

12 

1710 

(wrong  indentation  could  lead  to  programming  errors) 

1 3 

1710 

see  12 

1 a 

1720 

IS 

1725 

1 O 

1/31 

1030 

1 7 

1 050 

This  statement  was  inserted  netween  the  wronq  lines. 

IS 

1110 

It  was  detected  in  this  early  stade  of  the  oroject 
oecause  the  orooraomer  was  ounchino  caros  himself,  which 
allowed  him  to  review  11s  source  statements. 

Programmer  did  not  !oc«  at  the  declarjtion  of  the  vartaole. 

1=> 

1115 

20 

1 120 

(massing  ran datory  declaration) 

21 

1125 

(wrong  format) 

22 

1 130 

2 3 

1 l 35 

same  as  20 

2a 

1 1 ao 

25 

1/31 
13  35 

soecial  character  (")  was  enoected  to  Se  necessary  due  to 

26 

1335 

similarity  with  other  uses  of  * RITE/WRI T E 0 N * however  this 
assumption  -as  wrong 

(previously  declared  variable  declared  twice) 

27 

1335 

same  as  20 

28 

1 335 

2R 

1335 

Programmer  did  not  check  the  programming  manual. 

30 

1335 

see  20 

31 

1 335 

32 

1 335 
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ANNEX  E2 


Page  2 of  2 


ERROR  LISTING  (COMMENTS) 


ERROR 

ft 

DAY 

TI*E 

COMMENTS 

(EVIDENCE,  THGUGHTS,  why  nAS  THE  ERROR  v A DE ? 
WHY  AND  HO'.V  WAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 

1/31 

33 

1906 

34 

1 906 
2/04 

Error  could  have  been  avoided  by  loo<ing  at  orev'OuS  bloc*  o'  Code. 

35 

1 122 
2/05 

Porgra^mer  haa  co^p^ct  cesiqn  ccnceot  cuf  dig  cone 

cor rec  t 1 y . 

36 

1345 

3ouncary  conditions  •'ad  ngt  he«n  cnec»cea  ge*c  re. 

37 

1420 

39 

1420 

3» 

1 450 
2/05 

40 

1 445 
2/ 05 

Trivial  error  *as  because  srogra^m^r  «as  tired. 

41 

1500 

Design  -as  not  stated  well. 

42 

1530 

sane  as  40 

43 

1530 

2/0o 

44 

1525 
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ANNEX  F 


FINAL  STATISTICS 


PROJECT  a 1 


FINAL  STATISTICS 
Project  nai*«  : PALINDROMES 
Short  lescrict ion: 

Giver  an  inout  strinq  of  length  n,  ..here  2 c - n < c 25 0 » 
fino  all  occurences  of  oa' i»aroee?  anq  list  t'»n  »i tn 
reSDect  to  their  oeginnina  ana  ending  ccsitions  on  t'e  inout 
cerq. 

Inout:  via  ounch  cards 
Outcut:  via  line  orinter 

Guantitative  measures: 

1.  * of  source  statements  : lui 

2.  Total  man  hours  ‘or  orcject  : 21.8 

3.  Man  hours  scent  in 

a ) Design  : S . 0 

o ) Codi ng  : 7.0 

c)  DeOuggino  : 8.0 

d)  Testing  : 5.8 


a. 

CPU 

t i me 

for 

comoiles:  20.22 

sec  • 

5. 

CPU 

t i me 

for 

aeouq  runs : ° . 1 1 

sec 

o . 

CPU 

t i me 

for 

test  runs : 1 3 . R9 

sec 

7. 

a o» 

test 

ana 

oeoug  runs:  5 

8. 

a of 

test 

305 

deoug  steos:  23 

R.  » of  errors  found:  MR 

10.  Total  man  hours  used  to  correct  errors:  0.73 


- 7<5  - 


final  statistics 


11.  Error  Detection: 

a)  Mean  tine  between  error  detections:  20.3  nan  min. 


a of 
Errors 
t 


0 -15  lb-30  3 1 - J S 4b-o0  6 1 -75  76-00  91  -1  05  1 06-  1 30  15  1 -1  75  -nan  min. 

0 f 

oroject  time 

TIME  BETWEEN  ERR0®  DETECTIONS 

(measured  from  coin!  in  time  of  detection  of  orev'OuS  error) 
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ANNEX  F 


FINAL  STATISTICS 


12.  Error  Correct i on: 

a)  Mean  time  to  correct  an  error:  7.8  man  min. 

# o i 

Errors 

corrected 


2 0 + 


15  + 


10  + 


S + 


i : 


— - - — 

2 3 a 5 10 


15 


20 


> 

30 

■nan  t i n U t ?S 
C f project  t ' 


TIME  TO  C 0P°EC T ERRORS 
(neasured  fron  point  in  t i of  detection) 


t e 
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ANNEX  F 


FINAL  STATISTICS 


13. 

when 

errors 

were 

f ouna : 

a ) 

3 

of 

errors 

f ound 

during 

design  ohase: 

b) 

3 

0 f 

e r r o r s 

found 

during 

design  review 

c) 

3 

o f 

errors 

f ound 

during 

codi ng : 

a) 

9 

o f 

errors 

f ouna 

during 

deouga i ng : 

e) 

9 

o f 

errors 

f ound 

during  writing  of 

test  orocedures : 

f ) 

9 

0 f 

errors 

f ound 

du  r i ng 

testing: 

1U. 

when 

e r ro  r s 

were 

face : 

a ) 

9 

o f 

errors 

nade 

du ring 

design  ghase: 

o ) 

9 

O f 

e r ro  r s 

flaoe 

Our i ng 

design  n ev i ew 

c ) 

9 

o f 

e r ro  r s 

■ "aae 

du ring 

c o d i n a : 

d ) 

9 

o f 

errors 

face 

during 

deouga i ng : 

e ) 

9 

c f 

errors 

fade 

du  ring  * r i t i na  of 
test  oroceaures: 

f ) 

9 

o f 

errors 

fade 

a u r i r.  a 

test i na: 

5 

0 

se 

1 

o 

o 

uu 


2.3  X 
2.3  X 
50.0  X 
u 3 . 2 X 

0.0  X 

2.3  X 


n . j x 

0.0  X 
do.“  X 

2.3  X 

0.0  X 
0.0  X 
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ANNEX  F 


FINAL  STATISTICS 


15.  TIME  HISTORY  graphs  : 


» of 

Errors 

T 

« 

• 

! Error  Tyoes: 

! *!  Clerical  error 

! B:  Deougqinq  error 

! C 5 CoO l nq  error 

! 0:  DesiQn  error 

8 ♦ A 

! C 

: c 

5 ♦ C A 

: c c 

1 c c c cc  c 

1 C C C CCC  CA  C 5 

1 ♦ 0 0C  CCCCCCC  CCC  ACDCCCC  0 

* > 

! 2 i 4 5 10  15  80  ran  nouns 

..Design. ..Coding Debugging. .Testing.. 

(Software  Oevelocnent  0 h a s e s ) 

NUMBER  OF  ERRORS  FCUNO  VS  PROJECT  TIME 


* of 
Errors 

f 

5 ♦ * 

! * 

j * * * » 

{ * * * * *********** 

} * *****  ••«•  ************  * 

12  3^5  10  15  20  *an  nours 

MUM^ER  OF  ERRORS  CORRECTED  i S PROJECT  Tl*£ 


ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


1.  Design  Errors 

The  foilwing  tvoes  of  errors  acdy  To  both  categories 
"System  Design  Errors"  ana  "Program  Oesign  Errors": 

Dl  : Communication  Error 

D2  : Design  Negligence 

03  : Forgotten  Cases  or  Steos 

DR  : Timing  Problems 

05  : Errors  in  I/O  Conceots 

06  : Data  Oesign  Error 

07  : Initialization  Error 

08  : Inaoeouate  Checking 

OR  : Extreme  Congitions  Neglected 

D10:  Seauencing  Error 

Dll:  Indexing  Error 

DIE:  Looo  Control  Errors 

013:  Misuse  of  Boolean  Exoression 

014:  "atnematical  Error 

015:  Reo resen t a t i on  Error 

016:  yisuncerstanding  of  Proo'em  Soecifications 

017;  Other  Design  Errors 


2.  Coding  Errors 


Cl  : i sunae  r s t and  i rg  of  Design 

C2  : Neg I i gene  e 

C3  : 1/0  Format  Error 

C 4 : Mi solaced  Data  Declaration 

C5  : Mu I t i o I e Data  Declarations 

C 6 : fissmg  Oata  Declaration 

C7  : Inaoeouate  Oata 

C3  : Initialization  Error 

CR  : Error  in  Parameter  Passing 

CIO:  Iradeauate  or  Forgotten  Checking 

Cll:  Level  Problems 

C 1 2 : “issino  Declarations  of  31oc»  Limits 

C13:  Case  selection  error 

C14;  GO  TO  Prgolems 

C15:  Comment  Error 

C 1 6 : Forgotten  Delimiter 

C17:  Inconsistency  in  Naming 

C l 6 : Arong  Use  of  Nested  IF  Statements 

Cl°:  Indexing  Error 

C20:  Inconsistent  Use  of  Variables  or  Data 

C21:  Seauencing  Error 

C22:  Flag  Usage  Problems 

C23 : Syntax  Error 

C2R:  Looo  Control  Error 

C25:  Incorrect  Exit  from  Subroutines 

C26 : Language  Usaae  °ronlems 


ea 


J 
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ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Forgotten  Statements 

C28:  Representation  Error 

C2R:  Control  Seauence  Error 

C30 : Incorrect  Subroutine  Usaoe 

C 3 1 : Other  Coding  Errors 


3,  Clerical  Errors 

A 1 : Ma«ya 1 Error 

A2  : Mental  Error 

A3  : Procedural  Errors 

A u : Other  Clerical  Errors 


a . Oecugginq  Errors 


51  : Inacoroor  > at  e use  of  Oeouqoi''g  Tools 

B2  : Insufficient  or  Inaoorcon  ate  Selection 

of  Test  Cases  or  Test  Data 
53  t Misinteroretation  of  Debugging  Results 
84  : Misinteroretation  of  Error  Source 

55  : Negligence 

Bo  : Other  Debugging  Errors 


5,  Testing  Errors 

T1  : Inadequate  Test  Case ( s ) or  Test  Data 

T2  1 Misinterpretation  of  Test  Results 

T3  : Misinteroretation  of  Program  Sceci^cat>o^ 

T 4 : Neoligence 

T5  ; Otner  Testing  Errors 


es 


DIRECTED  GR4PH  REPRESENTATION 

PPCJECT  s : 1 

Program  cart  : INITIALIZATION 


COMPLEXITY  MEASL,pfS; 


NUMBER  OP  STATEMENTS:  is 


,\JM9£.A  CF  NCCES: 
NUMBER  CF  ARCS  : 


NUMSER  Cc  PATHS:  2 

CYCLC.MATIC  NUMBER:  x(S)  = 


o 


RE  AC  HAE I L ITY  CF  NCOcS: 


NQCE 
NCCE 
SCCE 
Nu  Cs 
NCCE 


c r r;  r 3 j . 


SUM:  7.CCCCCC 

REACHAEILITY  INDEX 
Dr  OIRECTEl  GRAPH: 


. • 400000 


© 


- 8b  - 


A,VN£<  G 


aoe  E c f A 


0IRECTE9  GRAPH  = £ppESE’.TA  r ICR 


PROJECT  = : [ 

Proqra"  ca'*t  ! b L A ' i K L I R F F 


CO“PlE*ITv  MEASURES: 


\ u w R F R OF  3 T A T E v E '-i  T 3 : o 

i\U  VF.  ER  CF  N C 3 E 3 : 6 

MJHEER  CF  ARCS  : a 

MJR9ER  Zr  RATES  z 3 

CYCLER  AT  IC  NUR3S.R  : /(C)  = 


REACHABILITY  CF  NCCES: 


A N h £ X G 


P a a e 3 


OIRECTF.D  graph  REPRESENTATION 

PROJECT  3 : 1 

Proaran  par*  : T £ '<  T 1 , T E Y T 2 * 1 c.  X T I 


COMPLEXITY  MEASURES: 

'lU'-'dEri  CiF  STATEMENTS:  A - II 

NUMBER  CP  NCOES  : a 

NUMBER  CP  ARCS  : 1 

NUMBER  CP  PATHS:  i 

CYCLE  VAT  IC  MJMSeS  s V l « J * l 


REACHABILITY  CP  N COES: 
iVCCE  1 : 1 

NUCe  2 : I 


SuM : 2 . CCCCCC 

REACHABILITY  1NCSX 


Z l°ECTED  G»AP* 


project  = : i 

P r o Q r a oarr  : p£AC  ANO  .'°IU 


CO^Pt-Ex  IT  < ASijRES: 


'lUMo 


t-i  CP  5T  i TE'-E'i  T S : IS 


Mjl*S  cS  Zr 
NO  PS  cP  CP  A * - . 

No  *6  2 5 CP  P A i P - * - , . 

CVCCCVAT  1C  .\o.v2  = ? : V(->- 


RcJCt-ASii  :ty  C ' 
NCCi  L - 

\ol  = l 

\ CCS  2 

NCCi  5 

NCCE  : 

NCCE  a 


NOCcS : 
1 


2‘ .CCCCC 


RsACrAiiLm  INCEX 


:F5P 


r.pc 


Ni'lC.  < 0 


r*  3 T p 5 n f ^ 


OHECTEO  w£?wP5F'!Ti  f Un 

PROJECT  « : 1 

Droqraffl  oa  r r • •** ^ I T E ^ L L 3 ^ L I ,N' ^ P 0 ^ E S 


a)  a ^ * ncces:  15 

c)  a of  arcs  : I*5 

c)  3 3 f state^enrs: 

3 ) a 3 ? car's:  * 

e ) 3eac^aotl»:v.’  * 
M CvC  I 0T3MC 


rujiciar  ! o 


A N .'I  c < G 


OISI-CTED  GRAP-i  PE 


PROJECT  = : 1 

Proorai  03  rr  I PALt'tOPQ^E  C^EC^ 


a ) 

3 Of 

nofie  S 

b) 

3 0 f 

arcs 

c ) 

= 0 r 

SMt?1 

a ) 

3 o * 

car's 

e ) 

3 e ac 

iaci  1 i 

*) 


°Eo£'i  TA  7 I G N 


c I c * 1 d *■  » c n u^cer 


htNiE  < 


PROJECT  s : 

Proorjip  sarj 

3 ) s o f nol 
C ) 3 of  a rz 
C ) 3 0 T S t 3 

a ) 3 of  cat 
e ) - e a c " a c i 
f ) C v c I o n a t 
* 'iuioef  of 
ana  react 


DIRECTED  ;P-P~  RE^RESEMAT  I C*i 


1 


: RECORD  °alIm)RO >’E 


es:  R 
s : i 2 
c e «*  • n c $ : c t 
ns:  * 

1 i r y : • 

ic  nutcer;  5 
o a t n s 


ao  > H / 


r 


ANNEX  H 


Page  1 oT  2 


TEST  PHASE  DESCRIPTION 


Project  « : 1 

Test  run  # : 1 Including  7 Test  Steos 


3eg  i 

n of 

Test 

(dav 

tine)  : 

2/06/ 1 a 30 

End 

o f 

Test 

(day/t i me ) 

CPU 

t i me 

for 

necessary  comoi'es  (in  sec.): 

: 5. 

18 

a ) 

1 .0* 

b J 

1.71 

c)  1.78 

a)  e ) 

f ) 

g> 

CPU 

t i me 

for 

TEST 

run  (sec) 

: 13. PB 

Man 

Hours 

i n 

Tes  t i 

no  : 5 • 9 

(including 

orecar at 

i on 

at  tests) 

2/06/1620 


TEST 

STEP 

OBJECT  V£ 

EXPECTED 

RESULT 

(TOLERANCE) 

ACTUAL 

result 

1 ) 

ERR0P ! CAY 
k : T IVE 

s 

t 

COMMENTS 

AND 

CODED  ERRGR 
types 

1 ) Record  -nen 

error  occurs 

1 

CnecK  croqran  for 

AM  oa  M no  romes 

2/06 

small  oa 1 i ndromes 

are  ce t ec  t eo  a^a 

1430 

(a)  Dal  indrome  of 

recorded  as  see- 

length  2 

cified  bv  or  o- 

Cb)  oal indrome  of 

oram  scecifi- 

1 enqt  h 3 

cat  ions 

O.K. 

2 

Check  Tor  large 

oa 1 i dromes 

same  as  1 

( a ) oa 1 i ndrome  o f 

o roqr . 

JU 

1525 

OR 

1 enot  h 255 

error 

( b ) oali nd  rome  o f 

1 engt  ft  25o 

3 

Test  correction 

Peoeat  steos  1.2 

same  as  1 

i ob 

1 550 

Imol ementat ion 

t ermi - 

error  (time 

mated 

estimate  exceed 

a 

Test  correction 

Peoear  s t eps  1 > 2 

same  as  1 

O.K  . 

5 

Check  oali ndromes 

should  be  de- 

wnicn  cross  cara 

dec  t ea  ana  o r i «- 

boundar i es 

ted  as  soec i f i ea 

0.K  . 

6 

CnecK  for  oal in- 

same  as  5 

dromes  o*  yarious 

1 engt  ns  and  sizes 

O.K. 

7 

C*ec  for  invalid 

■Naming  and  ter- 

See  notes 

i nout  ( s t r i ng 

m i na  t ion 

O.K. 

2/  Ob 

1 enat  h = 1 ) 

1 e20 

Notes:  Other  invalid  inouts  n*r;  exhaustively  tested  Ourinq 

deougoino  ohase. 
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ANNEX  H Pa0*  2 Of  2 


Choice  of  Test  Data: 

Some  test  data  were  choseri  to  check  the  program  for  boundary 
conditions.  Other  test  data  were  selected  to  check  random 
sizff  and  numbers  of  palindromes. 


APPENDIX  0 


PROJECT  DESCRIPTION 

Project  3 : 2 

Project  title:  PATH  ANALYSIS  IN  DIRECTED  GRAPHS 

Programmer  ; HOFFMANN 
Programming  Language  : ALGOL 

Programmino  environment:  ISM/360/fj7»QS/MvT / BATCH 

and  TIME  SHARING  (CP/CMS) 

Design  notes  : see  ANNEX  A 

Program  listing  : see  ANNEX  H of  APPENDIX  C 

Coding  notes  : see  ANNEX  C 

Debugging  notes  : see  ANNEX  D 

Error  Listing  : see  ANNEX  E 

Final  statistics  : see  ANNEX  F 

Graphical  representation  : see  ANNEX  G 

Test  phase  descriction:  see  ANNEX  H 

Starting  date:  1 77  Enoinadate:  17 

EXPERIMENT  DESCRIPTION 
1.  Project  description: 

A.  General  Description 

This  program  is  oesiqned  to  find  and  write  all 
paths  of  minimal  lenoth  (i.e.  if  a directed  qrarh 
1 ooD5>  no  looo  mav  be  traversed  more  than  once  i 
sion)  through  a directad  graph. 


mar  77 


possible 
contains 
n succes- 


9 7 


Input  Description: 


( a ) Batch 
Cara  1 : 1 


Cara  2:  t 


Card  3 : 


C a r d u : 


Processing  : 

in  card  column  1 followed  hy  any  te 
(this  text  will  identify  the  graph 
be  printed  as  a headline  preceding 
document  at  ion) 

in  card  column  1 followed  by  one 
where  t specifies  the  type  of  oath 


x t 

and  will 
the  incut 

b 1 a n k , 

1 i S t i n g 


de  s i r ed . 

t=0  ->  all  oaths  are  listed  during  analysis 
( recommended  only  for  a very  large 
number  of  paths/  i.e.  more  than  500 
paths  expected) 

t is  nonzero  ->  paths  are  listed  after 
analysis  has  been  completed 
(any  additional  information  will 
not  be  interpreted  by  the  oroorarr) 
s in  care  column  1 followed  bv  one  blank 
where  s specifies  the  number  of  diaits 
of  the  largest  node  name  used  in  the  directed 
orach  and  1<=  s < - 3 

(any  additional  information  will  not  be 
interpreted  by  the  program) 

1 n si  s 2 ...  sn 

where  1 is  the  node  name  of  the  entry  node 


and  may  be  punched  in  any  column 


n is  the  number  of  successors 
and  0 <=  n <=  1 0 

si  (i=l»?»  ...  n)  specifies  the  name 

of  the  ith  sucessor  of  node  1 
Card  5 thru  Caro  m + 3 ; 

where  m specifies  the  number  of  nodes  in 
in  t^e  directed  graph 
Format ! j n si  s2  ...  sn 

where  j specifies  the  node  name  of  the  node 
being  described  on  this  card 
and  <■?<=  j <=  999;  j may  be  Punched 
in  any  column, 
n is  number  of  successors 
ana  0<=  n <=  1 J 

si  ( i = 1 , <?  ...  n)  specifies  the  name  cf 
the  iths  successor  (1  <=  si  <=  999) 

t*  n=0  ->  no  entry  necessary 

* GTE  ; 

(a)  All  entries  are  integers  and  must  be  separated  by  at 
least  one  blank. 

(b)  A1  1 information  about  any  node  must  be  on  one  card. 

(c)  Card  column  30  must  contain  a blank  character  or  an 
additional  blank  card  has  to  be  inserted. 

Card  9 + m ; 90999  followed  ny  at  least  one  blank 

(this  input  indicates  that  all  information  of 
the  oreceedina  directed  graph  has  been  input) 
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NOTE:  Following  this  description  any  nymoer  of  directed 


graphs  may  oe  specified  and  submitted  to  the  program.  After 
the  last  descriotion  cf  a directed  graoh  a termination  card 
has  to  be  added  which  contains  any  integer  different  from 
"1"  in  card  column  1. 


Samp  1 e Input: 

1 (FIRST  GRAPH ) 

0 TOGGLE 

2 MAX  NODE  NAVE 

1 1 2 

2 2 3 4 

3 0 

4 1 5 

5 0 

qqqqq 

1 (SECOND  GRAPH) 

1 TOGGLE 

1 max  NODE  NAVE 

12  3 2 

2 0 

3 1 a 

4 2 15 

5 0 

qqqqq 

-1  TERMINATE 


(b)  Inout  under  C P / C M S : 

The  inout  seauence  under  CP/CWS  is  similar  as  unoer 
batch  orocessina.  There  is  only  one  exception:  Any  other 

inout  than  the  inout  requested  by  fhe  program  will  terminate 
the  orogram.  The  user  can  only  inout  integers*  additional 
blanks  are  not  a 1 1 o w e 0 . 

Under  CP/CMS  instructions  are  disolayed  at  the  terminal 
which  makes  the  use  of  the  program  almost  set  f-e*ol anatory. 
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NOTE 


(a)  Login  with  5 ^ 0 k 

(b)  Follow  the  directions  of  the  program  orecisely. 

Cc)  Incut  only  inteqers. 

C . trror  Messages  : 

All  error  messages  are  sel f*exd  anatory. 

Errors  3/  8,  9,  10  refer  to  invalid  or  incomplete  incut . 

Errors  8 and  9 will  cause  the  program  to  terminate. 

Error  2 indicates  the  limit  of  the  oroaram.  If  error  2.  oc- 
curs it  is  most  lively  that  the  directed  graoh  has  an  infin- 
ite number  of  paths.  The  user  may  try  the  same  input  using 
"0"  as  input  on  the  second  card.  Error  1 will  only  occur  if 

more  than  30  nodes  have  been  specified  having  1 4 successors 

each. 

All  other  errors  will  indicate  an  error  which  has  net 
been  found  during  testing  and  debugging  or  will  be  due  to 

some  abnormal  usaae  of  the  program.  In  either  case  it  miaht 

be  possible  to  locate  the  problem  by  examining  the  program 
listing  or  retrying  the  program. 


2.  Programmer's  background 


a)  Experience  in  programming: 


Oct 

1970  - May 

1971 

Programming  counses 

May 

1971  - April 

1972 

Module  Programmer 

May 

1972  - June 

1979 

rtork  in  Test  and  Simulation  Depart 

men  t at  the 

naval  command  and  CONTROL  SYSTEmS 
COMMAND  (FEDERAL  GERMAN  NA'VY) 

Testinq  of  tactical  real  time  systems 
March  1975  _ Jan  1977  Student  at  the  NAVAL  POSTGRADUATE 

SCbOOL*  Monterey*  Computer  Science 


b)  Experience  in  testing: 

Two  years  of  work  in  testing  ang  simulation. 

c)  Experience  in  the  area  of  the  given  problem:  None. 


d)  Experience  in  the  pronamminq  language  being  used: 
Experience  over  a period  of  IB  months  in  more  than  10 
programming  projects.  (Total  number  of  source  statements 
produced  during  that  time  was  more  9000.) 


5.  Ds yc h o 1 og i c a 1 factors: 


a)  Did  the  programmer  like  the  project?  Yes. 

b)  bow  does  the  programmer  like  the  programming 


E 


2 


1 anguage? 


Favorite  programmi  ng  1 snquaqe. 

c)  was  the  programmer  satisfied  by  the  way  the  problem 
was  soec i f i ed? 

Only  minor  criticism. 

d)  How  aid  tne  programmer  like  the  programming  environ- 
ment? 

T»-e  facilities  (stuov  room#  card  punch  room)  were  not 
conducive  to  efficient  programming  because  of  restricted 
space/  bad  lightinq  and  noise. 

e)  Other  factors: 

The  recording  of  the  experiment's  data  during  the  pro- 
ject affected  speed  and  concentration  consiaerably. 
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Comments  on  Documentation 


For  the  documentation  of  each  software  development  phase  a 
special  documentation  term  has  been  oevelooed.  These  forms 
are  aes i qned  to  provide  a firm  guideline  for  the  experiment 
programmer  to  record  all  data  of  interest  for  subseauent 
error  analysis. 

- Begin  and  end  of  each  step  was  recorded  with  respect  to 
day  and  time. 

- Each  error  was  recorded  when  it  is  discovered.  The  error 
was  then  identified  by  a uni oue  error  numter  (1,2/...). 
Furthermore  the  time  of  discovery  and  the  error  type  (using 
error  tyces  listed  in  ANNEX  F)  were  recorded. 

- If  appropriate,  comments  about  error  discovery,  reason  why 
the  error  was  made,  etc.  were  documented  in  ANNEX  E2. 

- For  each  error  the  phase  in  which  the  error  was  made,  the 
phase  in  which  the  error  was  discovered  ana  the  time  spent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

- For  each  step  in  any  one  of  the  software  development 
phases  the  aay/time  of  begin  and  end  was  recorded.  In  addi- 
tion, the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  docuren t a t i on  of  the 
experiment  data. 


1 Ou 
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WORKSHEET  FOR  OESIGN  PHASE  and  DESIGN  REVIEW  PHASE  OF  PROJECT  * 2 


STEP 

9 

PROBLEM  ANO  PLANNED  SOLUTION 

. / 

ALTERNATE 

SOLUTIONS 

DAY 
T I VE 

MAN  • 

HOURS ! ERROR 

/step:  * 

COMMENT 

i 

Analyzing  structures  with 

3/01 

• 

1 

resoect  to  minimum  number  of 

1900 

• 

oaths. 

1 

The  minimum  number  of  oaths 

» 

includes  all  oossible  oaths 

• 

starting  at  trie  entry  node  (1) 

1 

and  ending  at  any  node  which 

' 

1 2 . 0 i 

has  no  successor.  The  minimum 

1 

number  excludes  all  oaths  ha- 

1 

ving  reoetative  traversals  of 

looos.  such  that  any  sequence 

• 

of  arcs  is  traversed  *0  re  than 

t 

once  in  a row. 

3/03 

2U0O 

1 

2 

Design  of  Data  Structures: 

3/04 

1 

> 

Det  a i Is 

1 no 

• 

• 

see 

3/04 

f 

• 

ftema  r < s 

1900 

1 

1 

Remarks: 

Two  freelists  are  used  to  suooly  storaqe  soace  to  keec  information  about 
oaf's  ano  the  structure  of  t "e  ur.oerlvina  directed  gr aoh , 


FREELIST  1 (HI) 

(RSI  Items  of  R bytes  each  (32  Bits)) 

Items 


used  for: 

2 Bytes 

V 

A 

f\J 

Bytes  > 

a) 

1 

u u l i. ^ s Sor 

Informat i on 

successor  name 

1 0 1 r next 

successor  J 

in  forward 

1 

• 1 

linked  list) 


b) 

1 

1 

successor 

list 

1 

! reach ao i 1 i t v oarent 

I at r successor  list  ! 

• ‘ ' 

c ) 

1 

1 

Header 
(Item  0 ) 

! 0 
1 _ __ _ 

!otr  next  free  item  J 
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Items 
usea  for: 


FREELIST  2 (‘*2) 

[20001  Items  of  8 Bytes  each  (6a  0its)l 


< 2 Bytes  > < 2 Bytes  > 


a ) 

Node 

Information 
( due  I i c a t e ) 


oo inter  to  original 
o t r to  sutces'iO  r 


node  name 

o?r  to  oreoecessor 


0 ) 

pat  hneade r 
I n f o r*T*a  t ion 


otr  to  previous  catnjoath  i nen r i t i c e t i on 
ctr  to  oeoin  o*  oatnjetr  to  next  catn 


d) 

F ree list 
Heeae  r 
CItem  0) 


pointer  to  original 

node 

name 

* of  successors 

otr 

sue  c e 

SSO  r 

list 

0 

— 

0 

0 

otr 

next 

t ree 

i ten 

- 1 06  - 
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Tyoieal  usage: 

a)  Original  node  with  successor  list 


1 

1 

d J i » 

_ _ i _ _ i 

i i 

k!  MIL 

o I 

n J Dt  r ----- 

■* i — — — — — i 

ahere : 

i,  j,  k are  node  names 
r reoresents  reacnaoilitv  of  node  i 
o is  tre  storage  location  »ithin  v2 
NIL  inoicates  the  end  of  a linxeO  list 

S)  OouOlv  linked  list  of  oats  readers: 


* — _ _ _ . 
i 

•ot  r 
o t r 

! i 
J otr  — 



— > 

1 otr  1 otr----- 

V V 

oeginninq  oeginninq 

of  cath  i of  oatn  itl 


,1here  i and  i ♦ 1 are  oath  identifications, 
c)  Noces  (duplicates)  oeinq  elements  of  a oath 


original  <--- 

----ot  r 

-» -»  1 
i i 

[ f oredecessor  of  node 

o f i 

» **  ■" 

1 • 

Otr 

« *" i "" 

« 

i 

• 

i 

• 

i 

ot  r--- 

t 

• 

i ■ • • 

. • « • 

•--ot  r 

• j < 

! ot  r 

• 

i 

i 

• 

1 pt  r--- 

V 


.•mere 


ana 


are  successive  nodes  on  a oat" 
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WORKSHEET  FOR  DESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  a 2 


STEP 

s 


PPOBLEM  and  PLANNED  SOLUTION 


ALTERNATE 

SOLUTIONS 


! h4n 

DAY  IHOURS 
TIME  1 /STEP 


ERROR 

8 


3 ITOP  - DOWN  Design: 

IDefine  tne  following  program 


3/05 

1500 


carts: 


-Data  structures 
-Primitives  to  suocorr  oata 


structures 


-Utilities  to  sucoort  Oata 
s t rue  t ures 

-Logical  orocedures  to  sucoort 
analysis  of  directed  graons 
-Suoroutines  to  sucoort  analy- 
sis of  oossiole  oatns  tnrougr. 
a d i rec  t ed  graon 
-Inout  of  data 
-Outout  of  data 
-Er-or  messages  and  relevant 
diagnostics  to  sucoort  error 
error  i n ves t i ga t i on 


: .0 


3/05 
1 oOO 


108  - 


COMMENT 


1 9 


1 


r 
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WORKSHEET  for  DESIGN  phase  AND  DESIGN  REVIEW  phase  OF  PROJECT  a 2 


MAN 

STEP 

PPOBLEM  AND  PLANNED  SOLUTION 

ALTERNATE 

DAV 

hours 

ERROR 

COMMENT 

» 

SOLUTIONS 

TIME 

/STEP 

a 

4 

Define  Data  Structures 

3/05 

a)  Use  Freelist  (HI)  of  051 

A bvte  woras  usea  to  orovide 
information  about  successors 
of  parent  nodes  (each  of  ma*. 
30  oarent  nodes  may  nave  uo  to 
la  successors) 

b)  Use  Freelist  (M2)  of  20001 
0 bvte  items  to  keeo  informs* 
about  original  .nooes<  oath 
Headers  and  oossible  oaf's 

1 bOO 

.5 

through  the  directed  graon 
bei ng  ana  1 vzed. 

(eacH  item  is  designed  to  Hold 

3/05 

4 inteaer  values  oetween  0 ana 

1030 

see  also 

65535) 

r?nar<s 
on  oaae  1 

5 

Design  of  orimitives  and 

3/05 

utilities  to  Suooort  data 
Structures 

a)  Primitives  for  Freelist  1: 
•Initialize  Freelist  1 
•Carl,  Cdr!  (retrieval  of 

1030 

5.5 

1 n t eger  values) 

-Setcarl,  Setcdrl  (set  values) 
•Allocate!,  Freel  (allocate/ 
free  elements  of  Ml) 

3/05 

2030 

b)  Primitives  for  creelist  2: 

3/0o 

•Initialize  Freelist  2 
*Car2,  Cor2  (retrieval  of 

1330 

i n t eoe  r values) 

-Setcar2,  Setcor2  (set  values) 
•Allocate2,  Free2  (allocate/ 

3/0b 

free  elements  of  *2) 

1500 

1 09  - 
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WORKSHEET  FOR  DESIGN  PHASE 


ANO  DESIGN  REVIEW  PHASE  OF  PROJECT 


4 


z 


STEP! 

a ! 


PROBLEM  ANO  PLANNED  SOLUTION 


ALTERNATE 

SOLUTIONS 


6 IDefine  Algorithms  for  Utili- 
Ities  to  support  structures 
losing  M I : 

1-Name,  Setname 

1-Brother,  iacorofher 

i-Son(i)  Cget  name  of  itn  son) 

• 

% 

7 [Define  Algorithms  for  Ut i I i • 
[ties  to  suooort  structures 
lusino  '■'S: 

!a)SuPoort  of  linked  list  of 
loath  heaoers: 

1-Ser  Path  10,  -Patn  ID 
1-LinkpatnCfcrwarO/hack-ard) 

1— Initial °atnlist 
1-Ne*t  oath,  -Previous  °ath 
16)  Suooort  of  oath  structures: 
1-Name  of  (retrieve  nooe  na-e) 

1 -Predecessor 
1 -Successor 

1-Numoe-  of  Successors 
1-L'inkinq  of  nodes  ( ‘orward  and 
1 Backward) 

1 -Duo  I i c a t e ( se t duplicate  nooe) 
1 -Addnode 

1 — 0 uo lieate  a Path 
!-I*ole"entation  of  alternative 
loath 


MAN 


: o a r i hours 

1TIME1/STEP 


ERROR 

a 


comment 


3/0o 

1800 


.5 


3/  0o 
1330 

3/07 

1000 


3/07 
1 c 0 0 


3 IDefine  Procedural  llaorithms: 
1-Remove  a Path 
1 -L i St  Path 
1-List  all  Paths 
1-Find  End  of  a Path 
1-Find  Original 


3/081  1 

15001  .5  ; 

1 • 

• 1 

1 1 

15051  i 17 

09  former 

3/081  1 

analysis 

1530  1 1 

ai 1 not 

1 1 
1 1 

i nc 1 ude 

1 1 

• I 

trivial 

case 

1 1 0 
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WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  a : 2 
Beginning  of  Coding  (day/time)  : 3/06/1500 
End  of  Coding  (day/time)  : 5/10/1730 
Man  hour*  : 2h  (including  punching  of  cards) 


1 ) 


COOING 

BEGIN  1 END 

oay/ti«e  : day/time 

PROGRAM  part 

EPRO®  J 0 A y 
a J T IME 

COMMENT 

I)  decora  *n«n  error  is 

detected. 

03/06/1500 

•Data  Definition 

3/Qo 

-P  r i «n  i t i v e s 

i 

1530 

C28 

•Error  Handling 

2 

15a0 

C28 

-Initial i 2 a t i 0 n 

3 

1 o2Q 

C23 

03/0o/ to30 

03/06/ 1630 

-Punching  cards 

a 

1 7ao 

C28 

03/06/1800 

03/06/1630 

-Utilities 

03/06/2030 

03/07/1 200 

-Utilities 

3/07 

8 

1 305 

09 

03/0?/ta00 

03/08/1000 

-Punc^ina  earns 

3/08 

(Utilities) 

10 

1000 

C 1 2 (hissing  BEGIN) 

l 1 

1005 

C28  (mi soel ling  of 

procedure  name) 

12 

1005 

C23 

1 3 

1010 

Clb 

la 

1025 

Cl 

15 

1030 

C 1 7 

16 

1030 

C 1 7 

03/08/1200 

03/08/1530 

-Coding  of  orom 

ceaural  suorou- 

18 

1 o 30 

0°  ( des i gn  di o not 

tines 

consider  removal  of 

03/08/1800 

first  ana  last  oath) 

03/08/1 R00 

•Codi  ng  of  orom 

cedural  SuOrou- 

03/08/2000 

tines 

03/09/2000 

■Pune King  cards 

19 

2130 

Oil  (faulty  des i an 

af  inde«  calculation) 

20 

2150 

CIO  (faulty  condition 

"Not  Equal"  instead  of  *=" 

03/08/2200 

1 1? 


r 


ANNEX  C Page  2 of  2 

WORKSHEET  FOR  COO  TNG  PHASE  OF  PROJECT  « : 2 
Seginning  of  Cooing  (dav/time)  J 3/06/1500 
End  of  Cooing  (dav/tine)  s 3/10/1730 
Man  hours  : 26  (including  punching  of  cards) 


1 ) 


COOING  ! PROGRAM  PART 

3EGIN  ! EN0  ! 

DAy/TIME  J DAY/TIme  | 

ERROR 

a 

oa  y 
TI^e 

COMMENT 

1 

1 

1)  Record  when  er'or  is 

1 

detected. 

03/0R/12O0 

! -Codi nq  o f I/O 

3/09 

! subroutines 

03/0^/1300! 

03/09/1300 

!-°’jnc*inq  cards 

• 

i 

39 

1990 

Cl  1 

03/09/1500! 

03/06/1500 

! -Cod i na  of  Log i • 

! cel  o roc  *du  r e s 

35 

1530 

09 

i 

36 

1615 

09 

03/09/1  730,' 

03/0R/2000 

1-Punching  cards 

1 

1 

90 

2030 

C20 

03/09/aioo : 

J3/09/210O 

1-Coding  of  oath 

! analyzing  a 1 qo- 

92 

2200 

09 

! r i t hn ( "Se t 

03/09/23001  oaths") 
• 

03/10/1000 

1 

1-Punching  cards 

3/1  0 

1 

au 

1090 

015 

03/10/1100! 

03/10/1100 

1 “Coding  of  1/0 

! suOrout i nes 

03/10/12001 

03/10/1630 

1-Punchinq  cards 

1 

1 

96 

1700 

09 

0 3/ 1 0/  1 7 3u 1 
1 
1 
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worksheet  FOR  DEBUGGING  phase 
PROJECT  * : 2 OEBUG  Run  * : 1 

Begin  of  Deoug  Run  (day/time)  : 03/06/1800 

End  of  Deoug  Run  (day/tine)  : 03/07/1400 

* of  Debug  Steps  i nc  1 • in  Debug  Run:  4 CPU  time  for  Debug  run  (sec):  1.82 
CPU  time  for  necessary  compiles  (sec)  : 4.33 
a)  0,9R  b)  1,76  c)  l«6d  o 3 e)  f)  g) 


^an  hours  for  thi3  Debug  Run  : 4.0  (including  orecaration  of  debug  run) 

* A N 1 ) 


STEP'.  PRQGR  A^ 
a ! ° AR  T 

• 

OBJECTIVE  AMO  EXPECTED 

PE  SUL  T 

actual 

pesult 

CAY ; HOURS 
TI-aEi/STEP 

ERROR 

3 

Cghmc-jTS 
AUD  CODED 

error  types 

1 

• 

1 ) Record  «nen 

1 

1 

error 

# 

1 

occurs 

1 ; p r i m i - 

Get  error  free  comoile 

3 cons i 1 e 

3/0b 

! t i ves 

errors 

1311 

1.3 

5 

A 1 

! ana 

0 

C23 

'.Part  of 

7 

A 1 

lutili- 

3/Ob 

{ties 

1 3 3 0 

2 ! 

Repeat  step  1 

O.K  . 

3/07 

1.2 

1 

• 

1110 

3 J P r i m i • 

Check  initialization, 

0.5 

! t i ves 

allocation  and  freeing  of 

1 

1 

items  in  ootn  free'ists. 

1 

• 

Chec*  parameter  imolemen- 

1 

1 

t at i on  . 

♦ 

1 

Check  writing  of  slant 

1 

• 

1 i nes  . 

1 

1 

(pointers  of  freelists 

uooer 

1 250 

3 

A 1 ( • * 1 " 

1 

1 

must  be  set  aporco r i a t e 1 v , 

half  of 

left  out 

1 

1 

allocated  elements  must  be 

a 1 located 

to  h i 1 e oun- 

» 

filled  with  zeroes , 

e 1 ement 

C h l ng  c a ras 

» 

1 

allocated  items  must  be 

of  M2  not 

1 

• 

u n 1 i n k e d , 

filled 

1 

1 

after  freeing  an  element 

with 

1 

1 

becomes  part  of  the  free- 

zeroes 

• 

1 

list  a ga i n. 

I 

• 

parameters  should  be  set 

1 

1 

as  designed) 

a ; 

reseat  steos  1 and  J 

O.K. 

3/07 

1.0 

i 

• 

1 u 0 0 
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WORKSHEET  FOP  DEBUGGING  PHASE 
PROJECT  a : 2 DEBUG  Run  « : 2 

Begin  of  DeOug  Run  (day/tine)  ! 03/08/1000 

End  of  Debug  Run  (day/tine)  t 03/03/1700 

# of  Debug  S t eo  s incl.  in  Oebua  Run:  2 CPU  tine  for  Debug  run  (sec):  . RS 

CPU  tine  for  necessary  conciles  <sec)  ; 2.23 
a)2.23b)  c)  o)  e)  f)  g) 


Man  hours  for  this  Debug  Run  : 2.S  (including  oreoaration  of  debug  run) 

RAN  1 ) 


STEP 

3 

PR0GRA  * 
PART 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL 

RESULT 

DAY  1 HOURS 
TIME • /STE= 

3 

CCMVE'J  TS 
A tNi  0 CODEC* 
EPROP  TYPES 

1 

1 

1)  Record  men 

1 

• 

e r rc  r 

1 

• 

occurs 

t 

P r i m i - 

Cnec<  nyTOfr  of 

3/03! 

t i ves 

allocations  in  both  free- 

1 00  0 ! 

c a * n ou  r s 

and 

1 i S t S . 

; o.3 

scent  in 

cart  cf 

(all  elements  •*ceot  free- 

O.K. 

• 

1 

c reoa  r at i on 

ut  i 1 i- 

list  headers  are  aMcca- 

1 

• 

of  aeouq 

t 1 es 

ted ) 

1 

1 

• 

run 

2 

Error 

Checx  error  -essages  for 

• 

t 

hanrj* 

exhaustion  of  each  free* 

3/08!  0.2 

1 i ng 

list. 

O.K. 

1 7 0 0 ! 
1 
1 
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WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  8 : 2 DEBUG  Run  8 : 3 

Begin  of  Debug  Run  (day/time)  ! 03/08/2200 

End  of  Debug  Run  (day/time)  ! 03/09/1300 

8 of  Debug  Steps  i nc 1 . in  Debug  Run:  7 CPU  time. for  Debug  run  (sec):  .6 


CPU  time  for  necessary  comoiles  (sec)  : 18.12 
a)  2.67  b)  2.75  e)  2.75  d)  2.43  e)  3.49  f)  4.03  g) 


‘••an  Hours  for  inis  Ceoug  Run  : 4.5  (including  creoaration  of  debug 

«AN  [) 


ST£P  J PROGRAM 

OBJECTIVE  A NO  EXPECTED 

ACTUAL 

DAY ! HOURS ! ERPOR 

a • PART 

i 

RESULT 

RESULT 

TIMEi/STEP 

- 

• 

i 

i 

• 

i 

• 

1 i P n i m i • 

Get  error  free  comoile  of 

75  c - 

t 

« 

• 

3/ oe ; 

1 t i ves / 

all  subroutines  and  ori- 

d i 1 e o i a- 

2200;  i.o 

! Error 

* i t i v e s e*clu<5ina  I/O  sub- 

gnosti c s 

• 

• 

! h a no- 

routines ana  logical  sub- 

( see 

2200; 

21 

il  i ng . 

rout  ines. 

renarKS) 

22 

1 Proce- 

23 

! dura  1 

1 

• 

24 

! subrou- 

1 

I 

llines 

1 

1 

1 

• 

• 

2 ! 

Reoeat 

s t eo  1 

2245;  0.5 

25 

1 

3 : 

i 

Peoeat 

s t eo  1 

2310!  0.5 
1 

2b 

• 

a ; 

Reoeat 

st  eo  1 

2323!  0.5 
1 

27 

5 ; 

Receat 

s t eo  1 

3/or;  i.o 

28 

i 

t 

1 030; 

29 

i 

i 

i 

i 

30 

i 

• 

i 

i 

31 

i 

i 

t 

• 

32 

• 

• 

i 

i 

• 

i 

33 

o ; 

i 

Reoeat 

s teo  1 

O.K# 

1130!  0.3 
1 

7 • 

Reoeat 

debug  runs  1 and  2 

0 • K • 

1150!  1.2 

• 

1 

• 

(same 

resu 1 t s e«oec  ted) 

3/09J 

1 

1 

Remarks : 

The  ALGOL,  comoiler  diagnostics  do  not  always  allow  a co»ol ete 
i nyes t i oat i on  of  all  errors. 


run  ) 

COMMENTS 
AND  CODED 
EPRCR  TYPES 


1 ) Record  «nen 
error 
occurs 


C2  3 
C 1 7 
C2  7 
A 1 


C 1 7 

A2 

C28 

A 1 
11 
At 
A 1 
A 1 
C27 
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WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  a : 2 DEBUG  Run  « : « 

Begin  of  Debug  Run  (aay/time)  : 03/0R/1500 

End  of  Debug  Run  (day/time)  : 03/10/1200 

a of  Debug  Steos  incl.  in  Debug  Run:  6 CPU  time  for  Oebug  run  (sec):  .61 
CPU  time  for  necessary  comoiles  (sec)  : 23. “5 
a)  3.67  b)  5.02  e)  5.20  a)  a. 60  e)  R.Q6  f)  g) 


Man  nours  for  this  Oebug  Run  : 3.0  (including  crecaration  of  debug  run) 

VAN  1 ) 


STEP ! RROGRAM 
a i PART 
1 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL 

RESULT 

day  j hours : ERP0° 

TI^EJ/STEP!  a 

• 

COMMENTS 
AND  CODED 
EfiPOR  TVPES 

f 

• 

1 ) Record  *nen 

• 

1 

error 

i ;ai  i 

Get  error  free  ca^oile. 

2 come i ' e 

3/5R 

occur s 

! suo  rou* 

e r po  r s 

1500 

0.5 

! t i nes 

i exceot 

1730 

37 

C23 

! 1 ogi cal 

38 

A 1 

! oroce- 

! aures 

2 : 

Recest  steo  1 

O.K. 

1730 

3 ; l/o 

Check  inout  for  trivial 

number  of 

1 R50 

0.5 

1 

1 

cases 

succes- 

2000 

3R 

C2o  (READ  and 

• 

• 

so r s o 1 a- 

READON  eon- 

1 

1 

cea  in 

fused) 

1 

1 

wrong 

1 

field 

u ; 

Reoeat  steo  3 »ith  trace 

2130 

0.5 

• 

i 

and  oreceainq  initialize- 

• 

• 

tion  of  freelists  as  »ell 

• 

• 

as  i n i t i a 1 i z a t i on  of  list 

i 

i 

of  oath  headers. 

wrong  va- 

2200 

u l 

A 1 

• 

i 

Check  function  "SOW  for 

lue  re* 

i 

• 

correct  return  values 

t umed 

• 

(trace  snould  matcn  * i t n 

f ro<"  "Nun 

• 

• 

design  considerations) 

ber  of 

• 

• 

Succes- 

• 

• 

sors" 

s : 

Reoeat  steo  4 

Mi ss i no 

2320 

0.5 

• 

i 

1 ink 

2340 

R3 

CR 

6 ! 

Reoeat  steo  4 

2350 

0.5 

• 

i 

O.K. 

i 

i 

3/10 

i 

i 

1 100 

Remarks:  Functions  and  subroutines  are  first  ceckeo  for  trivial 
cases  and  boundary  conditions. 


117 


ANNEX  0 


Page  5 o f 11 


WORKSHEET  FOR  0E8UGGING  PHASE 
PROJECT  » : 2 DEBUG  Run  * : 5 

Begin  of  Deoug  Run  (day/time)  : 03/10/1600 

End  of  Debug  Run  (day/tine)  : 03/10/2400 

* of  Debug  Steos  inel.  in  Debug  Run:  5 CPU  time  for  Debug  run  (sec):  1.35 
CPU  tiire  for  necessary  compiles  (sec)  : 29.49 
3)  5.71  o)  5.94  c)  5.43  o)  5.79  e)  t>.o2  f)  g) 


far  hours  for  this  Debuo  Run  : 5.0  (includino  creoaration  of  debug  run) 

9 AN  1) 


STEP ! PROGRAM 

s : rart 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL 

RESULT 

day! HOURS 
TIUE I/STEP 

SRPGP 

2 

1 

A 1 1 

P r i m i • 
t i ves 
and 

ut  i li  • 
t i es 

Check  for  trivial  incuts 
for  all  orimitives  and 
utilities. 

l error 
in  1 i n k - 
i ng 

3/10 

1530 

1 600 

0.5 

45 

2 

Reoeat  st»D  1 

1 error 

1730 

1 .0 

47 

3 

Heoeat  steo  1 

3 orooran 

errors 

found 

1930 

1030 

2.0 

48 

2100 

49 

2130 

50 

4 

Reoeat  steo  1 

2 program 
errors 

2200 

2250 

1.0 

51 

2300 

52 

5 

Reoeat  steo  1 

O.K. 

3/10 
2 330 

0.5 

COMMENTS 
AN0  CODED 
ERROR  TYPES 


1 ) Record  .nen 
error 
occurs 


C27 


09 


C21  (state- 
ment s i n re- 
verse order) 
D12  («rong 
stoooi ng 
Condi t 1 on ) 

C 8 (faulty 
initial ization 
o f 1 ooo ) 


94  (error 
in  correct- 
i ng ) . 

C24 


Remarss:  Procedures  checved  for  trivial  cases  and  boundary  conditions. 


ANNEX  0 


Page  6 of  11 


'fan  Hours  for  t n > 3 Oeoug  5un  : o.O  ( i ne ' uai nq  oreoaration  of  aeoug  run) 

MAPI  1 ) 


TE?  1 PQ0GRA* 

OBJECTIVE  AND  EXPECTED 

actual 

DAY : HOuPS ! ERPQP 

COMMENTS 

# : past 

RESULT 

RESULT 

TIME : /STEP 

2 

AND  CODED 

! 

eprop  types 

• 

1 ) 9ecor3  <•  n en 

• 

• 

error 

1 ! A 1 1 

Get  error  free  co^oi 1 • of 

B c omo i l e 

3/10 

1 .0 

occurs 

• 

• 

all  orogram  carts. 

errors 

2300 

1 

1 

2310 

53 

A 1 

1 

• 

Su 

A 1 

• 

55 

A 1 

1 

• 

5o 

A 1 

• 

1 

57 

A 1 

( 

1 

53 

C2S 

t 

50 

CP 

• 

1 

eO 

CP 

• 

2 : 

Peoeat  steo  1 

o.<. 

2330 

0.5 

• 

i 

3/10 

t 

• 

2«00 

i 

3 ! Loaf  cal 

Check  for  trivial  cases 

3/11 

3.0 

! DPoce- 

while  analyzing  one  oat** 

1 error 

1 J00 

! du  res 

Check  writing  of  a oatr. 

f ouno 

1 <400 

6 1 

C27  (reset 

! ana 

(all  functions  snoula  oer- 

f 0 rgo 1 1 en  ) 

J « r i t i nq 

form  as  aesianea) 

1 0 f a 

loath 

a ; 

Peoeat  steo  3 

2 errors 

15u5 

1.0 

• 

• 

f ouna 

1620 

62 

CIO  ( » rong 

• 

• 

stopoing  con* 

• 

t 

d i t i on ) 

• 

« 

1630 

63 

C 30 

• 

5 ! 

Peoeat  steo  3 

1630 

0.5 

• 

O.K. 

3/1  1 

• 

• 

1 TOO 

ANNEX  0 


Page  7 of  1! 


WORKSHEET  FOR  DEBUGGING  phase 
PROJECT  * : 2 DEBUG  Run  a : 7 

Begin  of  DeDug  Run  (day/time)  : 03/11/1700 

End  of  Debug  Run  (day/time)  ! 03/12/1600 

« of  Debug  Steos  inel.  in  Debug  Run:  5 CPU  time  for  Debug  run  (sec):  2.61 
CPU  time  for  necessary  comoiles  (sec)  : 38.79 
a)  8.05  b)  7.81  c)  7.88  a)  7.83  e)  7.22  f)  g) 


fan  hours  for  this  Debua  Run  : 7.0  (including  o'tcarat io«  of  debug  run) 

MAN  I ) 


3TE° ! PROGRAM 
a : PART 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL 

RESULT 

0 A f jHOuRS 
T I«E ; /STEP 

EPROP 

3 

COMMENTS 
AND  C0CED 
ERROR  TfPES 

1 

| 

1)  Record  Rnen 

1 

• 

error 

1 IProce- 

Chec*  analysis  of  oatnes* 

2 program 

3/1  1 

0.5 

occurs 

! dure 

(All  oathes  are  imolemen- 

errors 

1700 

: "Set 

ted  inoouolv  linsed  lists) 

1730 

t u 

D 1 2 ( m ronq 

1 oa  t h s “ 

stooping  con- 

1 

d i c i on ) 

1 

1 

1 7 4S 

65 

Dll  ( w rong 

1 

• 

i nee* i ng) 

• 

2 i 

Reoeat  steo  1 

1 error 

1805 

0.5 

1 

found 

1310 

bo 

C 1 1 ( same  as 

1 

f 

error  65  ) 

1 

3 ! 

Reoeat  steo  1 

5 errors 

1920 

3.5 

1 

found 

t 

3/12 

1 

l 

1 300 

67 

03  ( forgot  ten 

• 

■ 

1315 

b( 3 

D3  update 

1 

1320 

oR 

03  of  a van- 

1 

able) 

1325 

70 

C2R 

■ 

1 340 

71 

94  (si  sinter* 

oretation  of 

* 

error  source) 

4 ! 

Reoeat  steo  1 

1 error 

1400 

1 .5 

72 

DR 

i 

found 

5 i 

Peoeat  steo  1 

0.K  . 

1530 

0.5 

t 

3/12 

• 

1 

1600 

120 


ANNEX  0 Page  8 of  11 

WORKSHEET  FOP  DEBUGGING  PHASE 
PROJECT  « : 2 DEBUG  Run  » : 8 

Begin  of  Oebug  Run  (day/time)  ! 03/13/1600 

End  of  Debug  Run  (day/time)  : 03/ia/1600 

# of  Debug  Steos  i nc 1 • in  Debug  Run:  3 CPU  tine  f op  Debug  run  (sec):  162,75 
CPU  tine  for  necessary  conoiles  (sec)  : IB. SR 
a)  6.63  o)  b.uS  c)  6.51  o)  e)  f)  g) 


'X an  hours  for  this  Deouq  Run  : 17.0  (including  oreoaration  of  aeoug  run) 

a A N 1 j 


STEP 

PROGRAM 

OBJECTIVE  AMD  EXPECTED 

ACTUAL 

DAY;  hOLRS 

ERROR 

COMMENTS 

U 

PART 

RESULT 

RESULT 

T I VE 

/STEP 

3 

AMD  CODED 
ERROR  TYPES 

1 ) Record  «hen 

error 

occurs 

1 

A 1 1 

ChecW  various  small  direc- 

1 er-or 

3/13 

tec  qraohs  for  bouncarv 

f oung 

1600 

3.0 

COnai t i ons • 

in  *^at - 
c r i oq" 

2000 

73 

C2R  (wrong 
oranCiing) 

2 

Reoeat  stao  1 

1 error 

3/lo 

6.0 

f oung 

070  0 

( some 
oat  hj 
s k i oced 
Our i ng 
ana  1 ys i s ) 

1100 

7U 

CIO 

3 

Reoeat  steo  1 

O.K. 

1300 
3/1  a 

3.0 

1600 

Remarks:  Error  analysis  durinq  this  deoug  run  »as  *itre»eW 

difficult  and  tine  consumino  for  the  fol'owinq  reasons: 
- “ost  results  «ere  verified  oy  desk  checking. 

• The  i no  1 enen t a t i on  of  oaths  «as  checked  oy 
examining  the  contents  of  allocated  iters  of 
both  freelists  (M1»M2). 

* Sone  structures  being  tested  naa  a very  large 
number  of  oaths  ( > 2000). 

“ The  orogram  contains  a great  number  of  rather  complex 
algorithms*  eacn  of  whien  had  to  be  tested  with 
resoect  to  its  boundary  conditions. 
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ANNEX  0 


F age  9 o f 11 


Jt 


nORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  « : 2 DEBUG  Run  a : 9 

Begin  of  Debug  Run  (day/tine)  1 03/13/1100 

End  of  Debug  Run  (day/tine)  : 03/15/2300 

# of  Oebug  Steps  incl.  in  Debug  Run:  1 CPU  tine  for  Ceoug  run  (sec):  3.R2 

CPU  tine  for  necessary  comoi'es  (sec)  : 7.52 
a)  7.52  b)  c)  c)  e)  * ) <5) 


«jn  hours  for  this  DeCug  Run  : 3.0  (including  oreoaration  of  deoug  run) 

VAN  1) 


STEP ! PROGRAV 
a \ PART 
1 
1 

OBJECTIVE  AMO  EXPECTED  1 ACTUAL 

RESULT  ! RESULT 

1 
1 

cay; hou»s 
T I ME ! /STEP 

ERROR!  CCWE'iTS 
a ! AMD  CODED 
! ERROR  TYPES 

• 

• 

11)  Pec  o r o -n<»n 

• 

1 

1 

1 error 

1 

1 

« 

1 occurs 

i :aii 

Chec*  internal  error  5 

3/  1 a 

3.0 

• 

• 

messages.  1 

23a5 

(Error  messages  5*ot7#  ! 0.*. 

• 

i 

11.12,13  snould  be  orin.  ! 

• 

t 

ted)  1 

3/15 

i 

i 

1 

• 

2300 
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ANNEX  C 


Page  10  ol  11 


WORKSHEET  for  debugging  phase 


PROJECT  « : 2 DEBUG  Run  « : 10 

Begin  of  CeOug  Run  (oay/tine)  : 03/16/1500 

End  of  Debug  Run  (oay/tine)  : 03/16/1600 

8 of  Debug  Stecs  >nel.  in  Debug  Run:  1 CPU  tine  for  Oeoug  run  (sec):  6.75 
CPU  tine  for  necessary  conoiles  (sec)  : 7.73 
a)  7.73  o)  c)  g)  e)  f)  S) 


Han  Hours  for  this  Debug  °un  : 1.0  (incluaing  Drecarjtio"  of  oeouo  run) 

HAN  1) 


PROGRAM!  OBJECTIVE  AND  EXPECTED 
PART  I RESULT 


ACTUAL 

RESULT 


DAT ! HOURS ! ERROR 
TIhe:/3T£P!  s 


CCVHE';T5 
A NO  CODED 
ccoor  TVPE3 

) Record  «n 
• r»or 
occurs 


ANNEX  0 


°aqe  11  of  11 


WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  8 : 2 DEBUG  Run  a : 11 

Begin  of  Debug  Run  (day/time)  1 03/16/1600 

End  of  Debug  Run  (day/tine)  : 03/17/1200 

« of  Debug  Steos  incl.  in  Debug  Run:  2 CPU  tine  for  Oebug  nun  (sec):  6. as 
CPU  time  for  necessary  comoiles  (sec)  ! la. 65 


a)  7.60  b)  7.05  c)  a)  e) 


^an  hours  for  this  Debug  Pun  : 2.0  Cine 

udirig  oreoarat  ion  of 

'■*  AN 

ceoua  run) 
1 ) 

STEP 

a 

PROGRAM 

PART 

OBJECTIVE  AMD  EXPECTED 

RESULT 

actual 

RESULT 

DAY i wouRS 
T I«E : /STEP 

EPR0R 

COMMENTS 
AND  CODED 
ERROR  TYPES 

1 ) Record  «nen 

error 

OCC'jrS 

i 

A 1 1 

Check  error  message  s R. 

error 
nessaae 
a Q no  t 
orinted 

3/16 
leOO 
1 B 0 0 

1 .0 

75 

C2  1 

a 

Reoeat  steo  1 

O.K. 

3/17 
1 140 
3/17 
1200 

1 .0 

ANNEX  El 


of  J 


Page  1 


PROJECT  * : 2 


ERROR  LISTING 


Begin  cf  Project  (day/time) 
End  of  Project  (day/time) 
"an  hours  for  total  project 


03/01/1R00 

03/17/2200 

125.0 


ERROR 

a 

phase 

in  m « i c h 
ERROR  was 
a i s- 
covered 

PHASE 

in  Mh i c h 
ERROR  Mas 
■"ace 

ERROR 

TYPE 

(see  annex  F) 

TIME 

soent  to 
solve  me 
ERROR 
(m  an 
n i n . ) 

s of  OTHER 
STATEMENTS  OR 
PARTS  OF  ThE 
PROGRAM 
AFFECTED 

i 

Codi ng 

Codi ng 

C28 

c 

2 

Coding 

Codi ng 

C23 

s 

3 

Coding 

CoO 1 ng 

C23 

2 

4 

Coo l nq 

Codi ng 

C23 

1 

5 

Oebugg i ng 

Co ding 

A I 

5 

6 

Debugoi ng 

Cod i nq 

C23 

S 

7 

0»c  ugg i na 

Coding 

A 1 

s 

3 

OfCuqa i nq 

Cod i ng 

A 1 

10 

0 

Coding 

Design 

DR 

15 

ahole  algorithm  affected. 

1 0 

Coding 

Coding 

C 1 2 

1 

i l 

Cooing 

Coding 

C 2 8 

1 

12 

C o a i n q 

Cod » ng 

C2  3 

1 

13 

Coci ng 

Coding 

Clb 

l 

l« 

Codi ng 

Cod i ng 

Cl 

5 

IS 

Coo i ng 

Coding 

C 1 7 

2 

10 

Cod i nq 

Coding 

C 1 7 

2 

17 

De  3 i an 

Design 

DO 

t 

18 

Cooing 

Oes i gn 

DO 

30 

nhgle  algorithm  affected. 

1 o 

Codi ng 

Design 

0 1 1 

s 

20 

Cooing 

Codi ng 

CIO 

5 

21 

Oebugg i ng 

Cod i ng 

C23 

5 

22 

CeOuaa i ng 

Coding 

C 1 7 

15 

23 

Oebugai ng 

Coding 

C27 

5 

2u 

Deougg i ng 

Coding 

A 1 

2 

25 

Ceouggi ng 

Codi ng 

C 1 7 

5 

2b 

Debugoi ng 

Codi ng 

A2 

10 

27 

Debuggi ng 

Coding 

C 2 9 

10 

28 

Deeugo i ng 

Codi ng 

Cl  7 

5 

20 

Oebugg i nq 

Coding 

Cl  1 

5 

30 

Oebugg i ng 

Coding 

A 1 

15 

125 


ANNEX  El 


Paoe  2 of  5 


ERROR  LISTING 


PROJECT  a : 2 


Begin  of  Project  (day/tine) 
End  of  Project  (day/time) 
Man  hours  for  total  protect 


03/01/1900 

03/17/2200 

125.0 


ERROR 

« 

PHASE 
in  - n i c 6 
ERROR  -as 
a i s- 
coverea 

phase 

in  -n i c 6 
ERROR  -as 
nade 

ERROR 

type 

(see  ANNEX  F 1 

TIME 

spent  to 
solve  tie 
ERROR 
( Man 
n i n . ) 

a 0f  OTHER 
STATEMENTS  OR 
PARTS  QF  Th£ 
PROGRAM 
AFFECTED 

31 

Oeouaa i ng 

Codi ng 

At 

5 

32 

OeBuggi ng 

Cooi ng 

A 1 

2 

33 

Debuggi «o 

Coding 

C 2 7 

5 

3J 

Cooi ng 

Codi ng 

Cl  1 

1 

iifiol  e aJ  qori  tMi  aif  ectec. 

35 

Coa i no 

Des i gn 

09 

15 

<ihole  algorithm  affected. 

36 

Codi ng 

Design 

09 

15 

37 

Oebuga i ng 

Con i nq 

C2  3 

5 

38 

O+Duqq  i ng 

Cod i ng 

A 1 

5 

39 

Oeougqi ng 

Codi ng 

C2o 

30 

90 

Oecuggi ng 

Cod i nq 

C20 

5 

91 

Oeb’jgq  i ng 

Codi ng 

A 1 

5 

92 

Codi ng 

Design 

09 

20 

nnole  algorithm  affected. 

93 

Debuggi  no 

Oes i gn 

09 

10 

44 

Coding 

Design 

015 

5 

95 

Debuggi ng 

Coding 

C 2 7 

10 

46 

Coding 

Design 

09 

5 

97 

Oebugg i ng 

Design 

09 

60 

aho'e  subroutine  affected. 
(3  changes  necessary) 

98 

0*fcugQ i ng 

Codi ng 

C21 

10 

49 

Geouqn i nq 

Des  t gn 

012 

5 

50 

Oeouggi ng 

Codi ng 

C 8 

15 

51 

Oebugg i ng 

Debuggi ng 

99 

20 

52 

Oebugg i ng 

Coding 

C29 

5 

53 

Oeouggi no 

Coding 

A 1 

5 

59 

Debuggi ng 

Codi ng 

A 1 

2 

55 

Oeouggi ng 

Codi nq 

41 

2 

56 

Debuggi ng 

Coding 

41 

2 

57 

Oebugg i ng 

Coding 

A l 

2 

58 

Qebugai ng 

Codi ng 

C28 

5 

59 

Oebuqgi ng 

Codi ng 

C 9 

5 

60 

Debuggi ng 

Coding 

C9 

5 

12b 


ANNEX  El 


Page  3 of  3 


f 


I 

I 


f 


ERROR  LISTING 


PROJECT  * : 2 


9egi n of  Project 
End  of  Project 


(aay/t i me) 
Coay/t i me ) 


03/01/1900 

03/17/2200 


"fan  hours  for  total  oroject  : 125.0 


ERROR 

* 

PHASE 
in  wn i ch 
ERROR  *as 

di  S- 
coyered 

PHASE 
in  w h i c *> 
ERROR  * a s 
"ade 

ERPCR 

TYPE 

(see  ANNEX  F ) 

TI9£ 

scent  to 
so  lye  the 
ERROR 
( Ma" 
min.) 

u of  0Th£R 
STATEMENTS  cr 
parts  OF  Tm£ 

PROGRAM 

AFFECTED 

o 1 

Debuao i nq 

Cod  i nq 

C27 

30 

62 

Oeougq i ng 

Coding 

CIO 

1 0 

63 

Oeouqg i ng 

Codi nq 

C 2R 

1 0 

6 J 

Defcuga i nq 

Design 

012 

1 0 

65 

Oebuaqi na 

Oes i qn 

Dll 

5 

bb 

Oeouqq i ng 

Design 

01  1 

15 

67 

OeOuqq i ng 

Design 

0 3 

1 0 

68 

Deougg i ng 

Design 

03 

1 0 

6R 

Oebugq i nq 

Oes i qn 

03 

5 

70 

Oeougo i ng 

Coding 

C30 

15 

7 t 

DeOuqg i ng 

Cebuqq i ng 

84 

1 0 

72 

OeOuqd i ng 

Design 

09 

120 

73 

Oeouga i nq 

Coding 

C29 

180 

7 9 

Oeouqq i nq 

Oes i qn 

010 

5 0 

75 

Oeouggi nq 

Coding 

C21 

30 

127 


ANNEX  £2 


Pag*  1 of  3 


ERROR  LISTING  (COMMENTS) 


ERROR 

a 

OAY 

time 

COMMENTS 

(EVIDENCE/  THOUGHTS,  WHY  WAS  THE  ERROR  MADE? 
WHY  ANO  HOrt  WAS  THE  ERROR  DISCOVERED? 

ERROR  SLOCKING,  etc.) 

i 

03/06 

1530 

Errors  1,2,3  were  discovered  while  reading  previously 

2 

i5ao 

written  sections  of  cooe. 

3 

l o20 

a 

1 7«0 

Lac*  of  concent  rat i on  while  Bunching  cards. 

5 

1815 

(Main  aisadvantage  while  ounching  cards  is  that  ounchea 
data  is  not  immediately  seen  after  each  key  strone.) 
same  as  a 

6 

1815 

Programmer  did  not  cnecY  programming  manual,  (error  could 

7 

1315 

have  oeen  avoided) 
same  as  a 

3 

03/07 

1305 

9 

1305 

10 

03/03 

1000 

Errors  10-17  were  detected  oecause  oroarammer  punched  cards 

1 1 

1 005 

himself.  It  is  ouite  natural  that  he  uses  this  time  to  review 
h i s code  . 

12 

1005 

1 3 

1010 

1 a 

1025 

15 

1030 

1(9 

1030 

17 

1505 

18 

1630 

Error  found  durinq  desx  test. 

1 9 

2130 

Error  fovjno  while  ounchinq  caros. 

20 

2150 

same  as  1° 

21 

2200 

22 

2200 

Function  name  used  as  local  variaele. 

23 

2200 

2a 

2200 

25 

22«5 

Incomolete  correction  of  error  s 22. 

26 

2310 

Mandatory  declaration  emitted  while  punching  cards. 

27 

2323 

Right  oarentesis  omitted. 

28 

03/09 

1030 

LacY  of  concent  rat i on  while  Dunphinq  cards. 

2R 

1030 

(Programmer  was  tired.) 
same  as  28 

30 

1030 

same  as  28 

31 

1030 

same  as  28 

32 

1030 

same  as  28 
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annex  E2 


Page  2 of  3 


ERROR  LISTING  (COMMENTS) 


ERROR 

u 

DAY 

TIME 

COMMENTS 

(EVIDENCE,  THOUGHTS,  WHY  WAS  THE  ERROR  made? 
WHY  AMO  HOw  WAS  THE  ERROR  DISCOVERED? 

ERROR  SLOCKING,  etc.) 

33 

03/09 

1030 

34 

1440 

35 

1530 

Extreme  conditions  not  carefully  an alvreo. 

36 

1615 

sane  as  35 

37 

1730 

LacY  of  concent  rat i an  while  coding. 

38 

1730 

39 

2000 

"READ"  and  "REAOON"  confused. 

90 

2030 

Usinq  a local  variable  name  from  different  subroutine. 

9 l 

2200 

42 

2230 

43 

2340 

Use  of  wronq  oara^eter. 

44 

03/09 

1040 

Error  found  *hile  ounching  cards. 

95 

03/10 

1600 

46 

1700 

Thinking  of  Douncarv  conditions  while  ouncning  cards. 

47 

1730 

DesY  test  was  not  mage  carefully  enough,  (one  case 

48 

1930 

omitted) 

Two  statements  in  reverse  oroer. 

4R 

2100 

50 

2130 

Error  -as  discovered  during  Structured  .Nal<  TKrouqr. 

51 

2250 

Caused  by  changing  code. 

52 

2300 

53 

2310 

LacY  of  concentration  whi'e  ounching  cards. 

54 

2310 

(see  comment  » 4) 
same  as  53 

55 

2310 

same  as  53 

5o 

2310 

same  as  53 

57 

2310 

same  as  53 

58 

2310 

59 

2310 

Programmer  did  not  chec*  or?v>ou3  procedure  declaration. 

aO 

2310 

(faulty  invocation) 
same  as  5R 

61 

03/  1 1 
1400 

Forgotten  reset  of  parameter. 

62 

1620 

Inaooropr i ate  stoDoing  condition  in  looc. 

63 

1630 

(confuseo  with  similar  condition  in  a oreviuos  problem) 
wrong  use  of  a subroutine. 

64 

1730 

Error  found  usinq  a trace. 

o5 

1745 

same  as  64 

66 

1810 

Error  found  during  desY  test. 
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ERROR  LISTING  (COMMENTS) 


ERROR 

« 


DAY  i COMMENTS 

TIME!  (EVIDENCE,  THOUGHTS,  ANY  WAS  THE  ERROR  m4QE? 
! WHY  ANO  HOW  WAS  THE  ERROR  DISCOVERED? 

! ERROR  BLOCKING,  etc.) 


67 

68 
6R 

70 

71 

72 


73 

74 

75 


03/12 
1 300 
1315 
1320 
1 325 

1 340 

1400 


03/14 

2000 

03/15 

1100 

03/16 

1800 


Necessary  uooate  of  a variaole  left  out. 
sane  as  67 
same  as  o7 

Boolean  e<oressior  does  not  meet  the  needs  of  t "e 
algorithm.  Erfjr  found  by  tracing. 

M i s i n t ero r t a t i on  of  the  error  source.  Error  founo 
while  receating  debugaing  steo. 

Different  benaviour  o*  algoritm  for  removal  of 

last  oath  orevious  to  the  last  one  »as  not  considered. 

Error  founo  ov  des*  test  of  all  relevant  subroutines. 

Error  founo  during  desk  test. 


Error  founo  during  desk  test. 
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FINAL  STATISTICS 


PROJECT  s 2 


FINAL  statistics 

Project  name  : PATH  ANALYSIS  IN  DIRECTED  GRAPHS 
Short  aeseriotion: 

This  program  is  designed  to  Tina  all  possible  oaths  of 
minimal  length  (i.e.  if  a directed  oraoh  contains  locos  no 
1 ooo  may  be  traverseo  more  than  once  in  a row)  through  a 
girectea  graon. 

Input:  via  ounc'  carss  Output:  via  line  printer 
Quantitative  measures: 

1.  » of  source  statements  : 712 

2.  Total  man  hours  for  oroject  : 125.0 

3.  Man  hours  soent  in 

a ) Design  : 31.0 

b)  Coaing  : 2b. 0 

C)  Debugging  : 55.0 
0)  Testing  : 13.0 

u.  CPU  time  for  compiles:  206.35  sec. 

5.  CPU  time  for  oeoug  runs:  I3R.R6  sec. 
o.  CPU  time  for  test  runs:  36R.73  sec. 

7.  a of  test  ana  3ebug  runs:  13 

8.  a of  test  and  deoua  steps:  R3 
R.  B of  errors  found:  75 

10.  Total  man  hours  used  to  correct  errors:  16.5 


annex  f 


final  STATISTICS 


T * of  errors 

t 

as  ! » » 


11.  E r ror  Detection  : 

Hean  t i we  between  error  detections:  69  "an  "in. 


aO 


J5 


JO 


25 


20 


15 


ran  hours 
->  of 

o ro  j ec  t t • re 


ANNEX  F 


FINAL  STATISTICS 


12.  Error  Correction: 

Mean  time  to  correct  an  error:  13.2  man  min. 

a of 
Errors 
cor rec  ted 
t 


29 

25 

20 


15 


10 


5 


i;***  * * * * 

1 5 10  15  20  25  30 

TIME  TO  CORRECT  ERRORS 
(measured  in  from  coint  in  time  of 


60 


120  190 

man  mi nu t es 
of  oro  j ec  t t i 


detection) 
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FINAL  STATISTICS 


13. 

when 

errors 

were 

found  5 

a ) 

a 

o f 

errors 

found 

dur i ng 

design  ohase ; 

b) 

a 

o f 

errors 

found 

du  r i ng 

design  rev i e» 

c) 

a 

o f 

errors 

found 

du  r i ng 

coding: 

a) 

a 

of 

errors 

found 

aurinq 

debugoi ng : 

e) 

a 

of 

errors 

found 

du  r i ng 

wr i t i ng  of 

tesr  orocedures: 

f ) 

a 

0 f 

errors 

found 

during 

testing: 

la. 

when 

errors 

were 

made : 

a ) 

a 

o f 

e r r o r s 

made 

durinn 

aesiar.  erase: 

0 ) 

a 

of 

errors 

made 

dur i rg 

desian  review 

c) 

a 

of 

e r ro  r s 

naoe 

during 

coding: 

d ) 

a 

of 

e r r o r s 

wade 

du  r i ng 

deouaa ina: 

e) 

a 

o f 

errors 

mane 

during  writing  of 
test  orocedures: 

f J 

a 

0 f 

errors 

made 

du  ring 

test ina: 

1 = 1 .3  X 

0 = 0.0  X 

21  = 28.0  X 
53  = 70.7  X 

0 = 0.0  X 

0 — 0.0  X 

75 


20  = 2o . 7 X 
0 - 0.0  X 

53  s 70.7  X 
2 - 2.7  X 

0 = 0.0  X 

0 = 0.0  X 

75 
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FINAL  STATISTICS 


15.  T I VE  HISTORY  GRAPHS  : 


# of 
Errors 
T 


Error  Tyoes: 

A:  Clerical  error 
9:  Debugging  error 

C : Coding  error 
0:  Design  error 


CCC  C CCCC  C A 0 D CA  CD  0 

CC  A A DC  DO  OCAC  CD  CAOOCCDA  CD  0 C 

2u  SO  90 

NLIM6ER  OF  ERRORS  FOUND  VS  PROJECT  T I gE 


■♦->  nan 
1 25  hours 


9 of 
Errors 


r • * * * * * • *■**•********  • 


2 2«  DO 


80  112 


125  nours 


NUMBER  OF  ERROPS  CORRECTED  V 3 PROJECT  TIME 
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ERROR  CATEGORIES  AND  TYPES 


1 . Design  Errors 

The  foilwing  tyoes  o*  errors  aooty  to  both  categories 
"System  Design  Errors"  ana  "Program  Design  Errors": 

01  : Commun i c a t i on  Error 

02  : Oesign  Negligence 

03  : Forgotten  Cases  or  Steos 

DR  : Timing  Proolems 

05  : Errors  in  I/O  Concepts 

Do  : Data  Design  Error 

D7  : In t t i a 1 ' Ja t i on  Error 

08  : Inaoeauate  Checking 

OR  : Extreme  Congitions  Neglected 

010:  Seaueneing  Error 

01!:  I ndex i ng  Error 

012:  Looo  Control  Errors 

013:  “isuse  of  Soolean  Excression 

DIR:  vathematical  Error 

015:  Pec r es en t a t ' on  Error 

Dlb:  ‘<i  sunoer  s t angi  ng  of  °roolen  Soec  f 'cat  tors 

017:  Other  Design  Errors 

2.  Cooing  Errors 

Cl  : '•*!  sunderst  angi  ng  of  Design 

C2  : Neo  I i gence 

C3  : 1/0  Format  Error 

CR  : Misolacea  Data  Declaration 

C5  : Mu I t i o I e Data  Declarations 

Co  : hissing  Data  Declaration 

C7  : Inaoeauate  Data 

C8  : I n i t i a 1 i z a t i on  Error 

CR  : Error  in  Parameter  Passing 

CIO:  Inaoeauate  or  Forgotten  Checking 

C 1 1 : Leve I Proo I e*s 

C12:  Missinq  Declarations  of  Block  Limits 

Cl  3:  Case  selection  error 

CIA:  GO  TO  Problems 

CIS:  Comment  Error 

Clo:  Forgotten  Delimiter 

C 1 7 : Inconsistency  in  Naming 

C18:  arena  Use  of  Nesteo  IF  Statements 

C 1 R : Indexing  Error 

C20:  Inconsistent  Use  of  Variables  or  Data 

C21:  Sequencing  Error 

C22:  Flag  Usage  Problems 

C23:  Syntax  Error 

C2R:  Looo  Control  Error 

C25:  Incorrect  Exit  from  Suoroutines 

C2t>:  Language  Usage  Pnoblems 
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C27:  Forgotten  Statements 

C28:  Reoresentat ion  Error 

C2R:  Control  Seauence  Error 

C30:  Incorrect  Subroutine  Usage 

C31 : Other  Coding  Errors 


3.  Clerical  Errors 

A1  : Manual  Error 

A 2 ; Mental  Error 
A3  : Procedural  Errors 

A4  : Other  Clerical  Errors 


4.  Debugging  Errors 


91  : I n aoo roc r i a t e Use  of  Oeougging  Tools 

82  : Insufficient  or  I naoo r oo r i a t e Selection 

of  Test  Cases  or  Test  Data 

93  : “ismterDretanon  of  Deougaing  Results 

94  : wi si nteroretat ion  of  Error  Source 

85  : Neal i gence 

9b  : Other  Oeougging  Errors 


5.  Testing  Errors 

T1  : Inaoeouate  Test  Case(s)  or  Test  Data 

T2  : si nteroret  at  ion  of  Test  Results 

T3  : “isinteroretation  of  Program  Scec i f i c at i on 

T 4 ; Neg I i gence 

T5  : Other  Testing  Errors 


A N h t < G 


Pace  1 of  ?! 


< 


CtBECfEO  GRAPH  PFPPESE'iTAT  I C 

PPOJECT  a : Z 

Program  cart  : C AP  1 , C AB 2 , CDP1,C0P2,  5ETCAB1,SETCAB2, 

SETCOm,  SETCORZ,  F»Et!,  ER£E  ? . SPACE, 
INITIALIZE  all,  set  NA«E,  AGCERCT-cP, 

PRO  Trtf  A,  .NAHR  OF,  RAT-  '.AVC, 

GuMPE-  ur  SUCCESSES,  - - E " £ C E 3 SOP , 
SUCCESSOR,  SET  = AT->  IS,  JuT  •.£<’  »lr- 
-EAGER,  ••£<!  path,  rdf./ICS  =4T», 

LINK  RAT-  oiC',  L I n«  P A T» , lm-h  RCP/vA-C, 
L I ‘ Kc  AC<  , 0‘J  PLICATE,  AOC-OCc, 
alte  = 'at:,E  ..ay,  imtsathlIST 


NOTE:  AM  33  suorouCir»s  "eve  •->»  sa-e  strict  a.‘». 


A\Nt  < G 


? of  ! 


OIPECTEO  G = -'P«  -E  = = ?SE  .TATIPf, 

psojeci  a : a 

P roaram  ft  1 ALLOCATE!/  Ai_LC*CATF<; 


COMPLEXITY  -a£ASuSE3: 


•iLm«EP  OF  STATE'-E  iTS:  7 - 


MjMSES  CF 
\LYBEo  CF 
,SL  />BES  C c 
CYCLCMATIC 


VCCES: 

A s C 3 5 
e ATI-  S : 

>N  L M 6 E S ; 


/II)- 


3EACHA6IL  ITY 


NCC: 

SCCE 

VCCE 

'■CCE 


Sly:  a.CCCCCC 

3EACFA2IL'.  TV  !NCEX 
CF  CIAECTEC  GSA=r: 


l.CCCCCO 


9 

I 


A N N t * o 


-’ace  5 o * 2 


DIRECTED  OFP^ESENT  A MCN 

PROJECT  a : 2 

Proafa'’’  cart  I ERROR 


CO^PLExlTf  *EASu*ES: 


DIRECTED  C,PAP4  seas?  SE  i T a T IG'I 


PROJECT  3 : 2 

Pr oQra'*’  cart  ; DIAGNOSE 


COMPLEXITY  '-'EiSoPES: 


■|UMB£P  OF  STATE-f.TS:  12 

VJFSc?  CF  \CCES:  3 

NuY9==  CF  ASCS  : i 

iNUFEEP  C F PATHS:  A 

CYCLE  MAT  IC  N L V 9 E F : V ( G ) = 


5=  aCVA3 

I L I T Y 

NOCE 

* 

NC  CE 

2 

MjuC 

2 

NC  L : 

•4 

NGC5 

5 

NCCE 

t 

V.CCE 

1 

NC  LC 

0 

SUV  ! 

=SaCHA9  II  ITY 


A 


17 .CCOOC 

!YCcX 


3 


A i^M  £ < L 


DIRECTED  GPiP-t 

PROJECT  a : 2 

Proqra*  cart  : SOM 


COMPLEXITY  E 1 S L 0 E 5 : 


OF 

5 r a T E *1  E 'j 

Vo, *5  EP 

r r 

N C C z 3 i 

.Nc*2  £3 

r s 

a * : $ : 

\U  *6  =5 

C r 

Tr  S : 

C V C L C ,v 

At  1L 

a?  iCHAJ  I L 
l 

vCCE  l 

VCCS  2 

VC  Cc  A 

VoCc  r 

vCCS  s 

VCCE  7 


ITy  ce  VCCES: 
. 1 


xw,v:  3.0C3CCC 

5=  iCi-Ac  IL  ITY  I'lCEx 
CP  CIRECTEC  GPAPP: 


I.C6S71* 


r 


4NNtX  G Pace  o n*  2\ 

OloeCTEO  GWSP-  =EPR?SF';T1TIC*: 

PROJECT  a : 5 

Program  carr  1 PINO  PPIGIN41 


COMPLEXITY  *&AS'jSES: 


•iu«bEP  if  statements:  it 

i\U!*6  38  CP  NCCES:  0 

Ml ,»8eS  CriSCS:  s 

\0*8eS  CP  3 4 7 r S : •» 

CYCLCYiTIC  NL.«e  = S : MC)  = 2 


CF  ,\  C E 3 S : 


t 

2 


IC.COOOC 
INC  EX 

Cr  CISEC7EC  CSmSK:  I.cEeocC) 


XEJCt-AfiiLITY 
NCCE  I : 

VCCE  2 : 

NCCE  3 : 

NC.C3  m : 

NCCE  5 : 

NCCE  i 

ElP  : 

as jci-sa  il 


ANNEX  G 


0 I a F C T E 0 =EccFSF'iTat  I 


PROJECT  * : 2 

Prjqrai  oart  : DbPLICATP  C'JFPE'iT  PATH 


COMPLEXITY  m£A5UPES: 


(IJMI-JS  of  STATE Mt  iTS:  15 
NL“P£S  CF  NCOSS:  5 

•MJ  P 3 c F CF  ASCS  : ; 

NU‘P=F  CF  FATl-5:  2 

CYCLl.mATIC  NuNfiEF:  v(G)  = 


CF  x CC  z S : 

1 

l 

‘ 

L 


7 . C C 0 0 0 0 
INCSX 


FEACFA2ILITY 

nc:=  i : 

\CCE  2 : 

NCCE  2 : 

NCCE  A : 

\CCE  5 : 

SUf* : 

SEJCriJ  IL  ;ty 


AN*Jfcx  ri 


arp 


O 0*  <M 


0[P£C  fEn  GP*?*  cc.og:SE'.^T  ION 


ottQJECT  * : 2 

Proar-jm  cart  • R E '•  * 3 ^ E c ^ ^ ^ 


C0mOLEUTt  ••’ENSURES: 


NU^HgP  ?F  iT-TF'-E*] 
NURSES  C r .\CCSS: 
NUMBER  Cz  -**C S _ : 
M^BER  C F R - 1 r - i _ 
“vriiiyATIC  No^3z**J 


S : 4 5 
1 «2 
3 c 
72 

V ( G ) * 


3 


* u m r e r o * o a ? s 


ANMt  l G 


0 age  13  o 


DIRECTED  0Rf-Oh  PEPFESE'iT  A T ION 


PPOJEC T * : 5 

Ppograa  cart  : OCCURS  TaiCE 


Complexity  v,EasuPES: 
‘1UWEEP  OF  ST  A TE'-'E'iTS  : 


il 


nl.vE  E* 

CF  NC 

r c c 

11 

\0.*SE5 

CF  ARCS 

1 A 

,\  t1  v 8 ? 5 

CF  FA 

7 r i 

2 3 

cycle* 

ATIC  NGM3 

E ? • 

~ 7 ( 

3 E -C  h A 

aarv 

r Z 

N 2 

C E 3 1 

nl.:s 

l 

l 

NCCE 

2 

1 

\CCS 

7 

SCCE 

A 

5 

\C  Cc 

3 

- 

nc:h 

6 

6 

NCCE 

7 

7 

nc:e 

a 

7 

\CCE 

\CCE 

s 

iC 

1A 

i* 

nQ  Cc 

11 

2S 

5L M J EE . COCZC 

F 2 AC  r A S IL  ITY  INlcX 
-r  0 I SEC  TEC  GRAFF: 


o 


* El 


5.45-1545 


/ 


ANNEX  G 


L’l^CTc')  GPAPh  5EP&ES6‘:T  AT  10 


PRGJECT  a : 2 

Ppoqran  oa^f  • ChEC^HAC* 


a ) 

3 

0 * 

no  09  s : 13 

n ) 

r 

o f 

arcs  •*  I o 

c ) 

2 

of 

statement; 

o) 

3 

Of 

oat^s!  * 

e ) •♦e^c^aoi  1 i t v : * 

f)  C yc  ’ o*at  ’ c r ; 5 

* ‘j-j'T'oe^  of  oa  t ■,  s 


ani  reac  3 ao  i t i * v 


OlWfeCTEO  'WiPx  ->EPPES£\TiriC\ 


POQJECT  = : 2 

Program  oart  : S^vE  \*Ev0PY 

COMPLEX  ! r Y v£iSiJPES: 


0 f 3 F c r £ 


PROJECT  s : 2 

Proqrari  cart  : L 1 3 T Al 


COMPLEXITY  •••pisoRes: 


'lUMPEP  OF 
VJ  V6  EF  CF 
N o iM  6 S 0 C = 

state  '•’£  *4  r 3 : 10  - 

\ f C 5 5 : -» 

! 

aaCE  : A 

Vj  >S  ES  c- 

PATi-5:  2 

cyclc.mjt  :: 

njv5SA:  v(G)= 

- 

SE  AC  FAS  1 1 1 

tv  CF  \C0ES: 

NC  C E L 

: L 

Ns,wc  Z 

: 2 

VOCE 

.VCCE  <♦ 

: 1 
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TEST  PHASE  DESCRIPTION 


Project  » : 2 

Test  run  a ; 1 Including  1 Test  Steo 

Begin  of  Test  (day/time)  ! 03/17/1500  End  of  Test  (day/tine)  t 03/17/1900 

CPU  tine  for  necessary  conoiles  (in  sec.):  7.53 
a)7,53o)  c)  c)  e)  f)  g) 

CPU  tine  for  TEST  run  (sec)  : 2S.29 

yan  Hours  for  this  Test  run  : 3.0  (including  oreoaratign  of 
tests) 


1 ) 


TEST 

STEP 

OBJECT VE 

EXPECTED 
RESULT 
( TOLERANCE ) 

ACTUAL IERRCR ! DAT 
RESULT!  a !TIh£ 
1 • 

1 1 

• • 

CCM*"EN  TS 
AND 

CODED  ERROR 
TrPES 

1 

1 ) ^ecorg  wnen 

1 

1 

error  occurs. 

l 

Test  oerfornance 

13/17 

of  oropram  for  va- 

! 1500 

rious  oi re c t ea 

l 

1 

graons : 

l 

1 

a ) di rectea  graph 

3 

vaHo  oatns 

O.X  . 

1 

• 

9 nodes,  10  ares 

• 

1 

b ) d i rectea  graon 

3 

valid  oaths 

O.X  . 

1 

1 

o nodes,  7 arcs 

l 

• 

c ) di rec  t ed  grach 

a 

valid  oaths 

O.X, 

a 

1 

*>  nodes,  7 arcs 

1 

1 

d)di rectea  graoh 

7 

valid  oaths 

0.x. 

• 

f 

22  nodes,  25  arcs 

t 

1 

e ) di rec  t ed  araoh 

i i 

valid  catns 

o.x. 

1 

« 

22  nodes,  27  arcs 

• 

1 

f ) di rec  t ed  grace 

no 

valid  oath 

O.X. 

a 

( 

1 node,  1 are 

warn i ng  shou 1 d 

a 

t 

(self  1 ooo  at  node 

ae 

printed 

a 

• 

1 ) 

• 

• 

gldirected  graoh 

2 

valid  oaths 

O.x. 

• 

1 

2 nodes,  2 arcs 

a 

a 

n ) di rec  t ed  graoh 

1 

valid  oath 

O.X. 

a 

1 

2 nodes,  1 are 

13/17 

! 1900 

Renarxs:  Oirected  graons  suOnitted  for  testing  include  a variety  of 
boundary  conditions  ana  soecial  cases  enich  are  considered 
to  be  difficult  to  evanine. 
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TEST  PHASE  DESCRIPTION 


Project  * : 2 

Test  run  a : l Including  1 Test  Steos 

Begin  of  Test  (day/time)  ! 03/15/1200  End  of  Test  (day/time)  : 03/17/2200 

CPU  time  for  necessary  comoiles  (in  sec.):  6.83 
a)6.83o)  c)  d)  e)  f)  g) 

CPU  tine  for  TEST  run  (sec)  : 0U4.UU 

Han  Hours  for  this  Test  run  : 10.0  (including  orecaration  of 

tests) 


1 ) 


TEST 

STEP 

GBJECTVE 

EXPECTED 

RESULT 

(TOLERANCE) 

ACTUAL i error 1 DAY 
RESULT!  s 1TIVE 

1 1 

COUNTS 

AND 

CODED  ERROR 
7f°SS 

1 I 

1 • 

1 ) Record  - -«en 

1 • 

1 1 

error  occurs. 

1 

Find  limits  of  the 

P rqgr  am  sncu 1 d 

! 13/15 

oroqram  usinq  a 

list  a f f oaths 

i 11200 

ci rectea  graon  ha* 

until  f reel i st 

i t 

• 1 

v i ng  an  infinite 

is  e*  haust  ed . 

1 • 

1 • 

nymoer  of  oaths 

Uoon  terminat ion 

• 1 

1 t 

accorai nq  to  the 

of  the  croaram  an 

O.K.  I ! 

definition  oeing 

aocroor i ate  warn- 

A t 

1 • 

used  • 

1 ng  should  oe 

• 1 

1 1 

orinteo.  Such 

• t 

1 • 

that  the  user  can 

1 • 

1 1 

interoret  t"e  re- 

1 ( 

1 1 

suits  of  tne  natn 

• • 

1 • 

analysis. 

: ;3/i7 

1 12200 

Remarks:  Due  to  extensive  oeougging  crevious  to  testing  no  er'grs 
have  oeen  found  during  trie  testing  o«ase. 

Test  data  for  tnis  run  are  Close"  to  ingicate  1 i - i t a t i on s 
of  me  oroqram  oecause  of  me  finiteness  of  1 1 »e  ang  n»»or* 
avai 1 ao I e . 
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APPENDIX  C 


PROJECT  DESCRIPTION 

Project  8 : 3 

Project  title:  PATH  ANALYSIS  IN  DIRECTED  GRAPHS 

WITH  RESPECT  TO  REACHABILITY  OF  NODES 

Programmer  : HOFFMANN 

Programming  Language  : ALGOL 

Programming  environment:  IBM/360/67, OS/ MVT, BATCH 

Design  notes  : see  ANNEX  A 

Program  listing  : see  ANNEX  8 

Cooing  notes  : see  ANNEX  C 

Debugging  notes  : see  ANNEX  D 

Error  Listing  : see  ANNEX  E 

Final  statistics  : see  ANNEX  F 

Graohical  reoresentation  : see  ANNEX  G 

Test  phase  descriotion:  see  ANNEX  H 

Starting  date:  1 MAR  77  Ending  date:  17  MAR  77 

EXPERIMENT  DESCRIPTION 
1.  Project  descriotion: 

This  program  is  an  extension  of  oroject  8 2.  Tne  graon 

analyzing  program  as  develoDed  in  oroject  8 2 was  modified 

to  calculate  the  reachability  of  each  node.  A summary  of 

the  calculations  is  printed  at  the  end  of  the  analysis.  All 

features  described  in  project  8 2 will  be  uncharged. 


2.  Programmer's  background: 

a)  Experience  in  programming: 

Oct  1970  - May  1971  Programming  courses 
May  1971  - April  1972  Module  Programmer 

May  1972  - June  1970  work  in  Test  and  Simulation  Deoart- 

ment  at  the 

NAVAL  COMMAND  AMD  CONTROL  SYSTEMS 
COMMAND  (FEDERAL  GERMAN  NAVY) 

Testing  of  tactical  real  time  systems 
March  1975  “ Jan  1977  Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL*  Monterey*  Comouter  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  and  simulation. 

c ) Experience  in  the  area  of  the  given  problem:  None. 

d)  Experience  in  the  prooammina  language  being  used: 
Experience  over  a period  of  Id  months  in  more  than  10 
programming  projects.  (Total  number  of  source  statements 
produced  during  that  time  was  more  9000.) 

3.  Psychological  factors: 

a)  Dio  the  programmer  like  the  project?  Yes. 

b)  How  does  the  programmer  like  the  orogramming 

- 1 b 3 • 


A 


1 anauage? 

Favorite  programming  language. 

c)  lias  the  programmer  satisfied  by  the  way  the  problem 
was  spec i tied? 

Only  minor  criticism. 

d)  How  aid  the  programmer  live  the  programming  environ- 
ment  ? 

The  facilities  (study  room,  card  punch  room)  were  not 
conducive  to  efficient  programming  because  of  restricted 
space,  bad  lightinq  and  noise. 

e)  Other  factors: 

The  recording  of  the  experiment’s  data  during  the  pro- 
ject affected  speed  and  concentration  considerably. 


4 . Comments  on  Documentation 


For  the  documentation  of  each  software  develooment  chase  a 
soecial  documentation  form  has  been  developed.  These  forms 
are  designed  to  provide  a firm  guideline  for  the  experiment 
programmer  to  record  all  data  of  interest  for  subseauent 
error  analysis, 

- Begin  and  end  of  each  step  was  recorded  with  respect  to 
day  and  time. 

- Each  error  was  recorded  when  it  is  discovered.  The  error 
was  then  identified  by  a unique  error  number  (1/2,...). 
Furthermore  the  time  of  discovery  and  the  error  fyce  (using 
error  tyoes  listed  in  ANNEX  F)  were  recorded. 

- If  appropriate/  comments  about  error  discovery,  reason  why 
the  error  was  made,  etc.  were  documented  in  ANNEX  E2. 

- For  each  error  the  phase  in  which  the  error  was  made,  the 
phase  in  which  the  error  was  discovered  and  the  time  scent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

- For  each  step  in  any  one  o f the  software  development 
phases  the  day/time  of  begin  and  end  was  recoraed.  In  addi- 
tion, the  time  (in  man  hours)  for  each  step  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
experiment  data. 


— ^ 
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ANNEX  A 


Page  1 of  1 


r 


WORKSHEET  for  DESIGN  phase  AND  DESIGN  REVIEW  phase  OF  PROJECT  a 3 


PROBLEM  ANO  PLANNED  SOLUTION 


ALTERNATE 

SOLUTIONS 


“AN  ! 

hours : error 

/STEP!  a 


COMMENT 


Analyze  ways  to  compute  t*>e 
reachability  inoex  Tor  each 
node . 

•In  general  the  reachability 
index  of  a noae  is  incremented 
by  1 wnen  tne  rode  is  added  to 
the  current  oatn  or  out  as  the 
first  node  on  an  alternative 
oath. 

-The  reachability  of  nodes 
which  belong  to  a oath  wnicn 
has  to  be  removed  has  to  oe 
decremented  oy  1 uo  to  tne 
node  wnicn  nas  been  the  last 
branch  ooint  on  this  oatn  ex* 
cludina  the  branch  ooint 
i t se  1 f . 


Too-Oown  Design: 

a)  Define  additional  subrou- 
tines: 

- Ucicate  Reachability 
( i nc  remen  t Oy  1 I 

- Correct  Reachability 
(decrement  by  1 for  all  ore- 
vious  nodes  uo  to  and  exclu- 
dina  the  last  branch  ooint) 

- List  Reachability 
(output  of  results) 

b)  Define  necessary  chanaes 
within  current  version 

of  the  analyzing  program. 
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ANNEX  C 


Page  1 of  1 


WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  « : 5 
Beginning  of  Coding  (day/time)  ! S/20/2000 
End  of  Cooing  (day/tine)  ! 3/21/1200 
Man  hours  : « . 0 (including  punching  of  cards) 


COOING 

BEGIN  ! ENO 

OAY/TIME  1 DAY/TIME 


i) 

: program  part 

:er°or:day  : 

1 a ! T IME ! 
• 1 • 

COMMENT 


ANNEX  0 


Page  1 of  1 


WORKSHEET  POR  DEBUGGING  Phase 
PROJECT  a : 3 0E6UG  Run  « : t 

Begin  of  Debug  Run  (day/time)  : 03/21/1200 

End  of  Debug  Run  (day/time)  : 03/21/1520 

a of  Debug  Steos  incl.  in  Debug  Run:  5 CPU  time  for  Debug  run  (see):  41.13 
CPU  time  for  necessary  comoiles  (sec)  : 29.98 
a)  6.03  d)  8.U3  c)  7.65  o)  7.97  e)  f)  g) 


Han  hours  for  this  Debuo  Run  : 3.0  (including  oreoaration  of  debua  run) 

han  1 ) 


STEP ! PROGRAM | OBJECTIVE  AND  EXPECTED 

a J PART  ! RESULT 

1 1 

1 1 

actual 

RESULT 

day ; hours 
ti«e: /step 

ERROR 

a 

COMMENTS 
AND  CODED 
ERROR  TYPES 

1 1 

1 • 

1 ) Record  «nen 

1 • 

1 1 

error 

1 1 

occurs 

l ',-Get  error  free  comoile 

1 co»o i 1 e 

3/21 

1 .0 

2 

C 6 

1 subrou- 1 

error 

1215 

( forgot  ten 

i t i nes  ! 

dec larjtion 

1 and  | 

of  v a r i a 0 1 e ) 

! program  1 

.'changes  J 

• i 

2 ! 1 -reoea t s t eo  ! 

1 oro- 

3/21 

0.5 

3 

C24 

1 1 

* 1 

gram 

1315 

• I 

• 1 

error 

• 1 

3 ! i-reoeat  steo  1 

O.X. 

3/21 

0.3 

1 t 

1 1 

1 345 

1 1 

a ! 1-Checx  reachability  cal- 

uronq 

3/2! 

0.7 

a 

C21 

! Iculation  for  trivial  cases 

b r a n c h - 

1400 

• • 

1 • 

ooi  nt 

1 1 

C a 1 cu 1 a- 

1 • 

1 1 

? r on 

( • 

5 1 1-reoeat  steo  a 

O.K. 

3/21 

IT 

O 

« I 

f 1 

/ 

1450 

• 1 

• 1 

3/21 

1 • 

• i 

1520 

ANNEX  e 


Page  1 of  1 


ERROR  LISTING 

PROJECT  # 5 3 

Begin  o f Project  (day/time)  : 03/1R/1500 
End  of  Project  (day/time}  : 03/23/2300 
Man  hours  for  total  project  : 33.0 


ERROR 

* 

PHASE 
in  i c h 

ERROR  was 
d i s- 
covered 

PHASE  ! 

in  which! 

ERROR  «as ! 
naoe  ! 

ERROR 

TYPE 

(see  ANNEX  E ) 

T I ME  J 
spent  to  ! 
soNe  t h e { 
ERROR  ! 

(*an 

■nin.)  1 

b of  OTHER 
STATEMENTS  or 
PARTS  OF  the 
PROGRAM 
AFFECTED 

i 

Codi ng 

Coding  ! 

C1R 

10  ! 

2 

Oeeugq i ng 

Coding  ! 

C6 

5 J 

3 

Oeougqi nq 

Cooing  ! 

C 2 A 

5 J 

4 

Deouqai ng 

Coding  ! 

C21 

20  : 

03/21 ! 

1115!  Checxinq  code  while  ounchinq  cards. 

1215!  Pound  through  co»oil»r  diaanosties. 

1315! 

tUOO!  Pound  Ov  interoretinq  results  of  aeoug  run, 


ANNEX  F 


FINAL  STATISTICS 


PROJECT  « 3 


FINAL  STATISTICS 

Project  name  : PATH  ANALYSIS  in  DIPECTED  graphs 

*ITH  RESPECT  TO  REACHABILITY  CF  NODES 
Short  aescriotiar; 

This  project  is  an  (ttersion  of  Project  - 2.  The  grson 
analyzing  orggram  gevelooed  in  oroject  * 2 is  modified  to 
calculate  the  reacnaoilitv  of  each  inaivioual  noce.  At  the 
ena  oi  the  analysis  the  reacnaoility  inae»  of  each  node  is 
printed.  All  features  of  tne  orogram  ceinq  chanaed  are 
preserved. 

Inout:  via  punch  cards  Outout:  via  line  printer 
Quantitative  measures: 

1.  * of  source  statenents  : 70  (including  necessary  chances) 

2 . Total  nan  hours  for  project  : 33.0 

3.  "an  hours  soent  in 

a ) Design  : 7.0 

b ) Codi  ng  : A . 0 

e)  Debugging  : 3.0 
o)  Testing  : 1R.0 

A.  CPU  tine  for  compiles:  Ea,08  sec. 

5.  CPU  time  for  debug  runs:  At. 13  sec. 

6.  CPU  time  for  test  runs:  3RI.A8  sec. 

7.  3 of  test  and  deDug  runs:  A 

8.  * of  test  and  debug  steos:  8 
R.  » of  errors  found:  A 

10.  Total  man  hours  used  to  correct  errors:  0.7 


- 1 95 
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annex  f 


FINAL  STATISTIC, 


11.  Error  Detection: 

Mean  time  Between  error  detections:  43.3  man  min. 

« of 
errors 
T 

I 

5 m 

» 

I 

♦ 


1 


******************* 


1 *5  30  n 5 o 0 75  ° 0 105  120  1 35 

TIME  SETnEEN  ERROR  DETECTIONS 

(measured  from  ooint  in  time  of  detection  of  crevious  error) 


- — - > -nan  min  of 
pro j ec  t time 


12.  Error  Correction: 

Mean  time  to  correct  an  error:  10.0  man  min. 


# of 
Errors 
cor  rec  ted 
f 

5m 


1 ! • * * 

- ♦ 

1 5 10  15  20  25  30  35  40 

TIME  TO  CORRECT  ERRORS 

(measured  in  from  ooint  in  time  of  detection) 


♦ 

man  minutes 
of  oro i ect  time 


i 
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ANNEX  F 


FINAL  STATISTICS 


13.  When  errors  were  found: 


a)  * of  errors  found  during  design  ohase: 

b)  * of  errors  found  during  design  review: 

C)  * of  errors  found  during  eodinq: 

d)  * of  errors  found  during  debugging: 

e)  * of  errors  found  during  writing  of 

test  orocedures: 

f)  a of  errors  found  during  testing: 


0 = 0.0  X 

0 = 0.0  I 

1 = 25.0  X 

3 = 75.0  X 

0 = 0.0  X 

0 = 0.0  X 

4 


14.  when  errors  were  made: 


a)  4 of  errors  nade  during  design  ohase: 

o)  « of  errors  nade  durirg  design  review: 

c)  a of  errors  -"ade  during  coding: 

o)  a of  errors  "ade  during  deougqing: 

e)  a of  errors  «ide  durirg  writing  df 

test  orocedures: 

f)  « df  errors  "aae  during  testing: 


0 = 0.0  X 

0 a 0.0  X 
4 1 100  X 

0 a 0,0  X 

0 = 0.0 
0 a 0.0 

4 


ANNEX  F 


FINAL  STATISTICS 


15.  TIME  HISTORY  GRAPHS 


Error  Types: 
a:  Clerical  error 
8:  OeOuqq’nq  error 
CS  CoOinq  error 
0:  Oes i an  error 


1 ! C C C C 

1 2 3 4 5 10  15  20 

..  .DESIGN CODING. .DEBUGGING. .TESTING. 

(soft-are  development  prases) 


► man 

30  33  nour- 


NUMBER  OF  ERRORS  FOUND  VS  RROJECT  TI^E 


a of 
Errors 


It  c c c c 

12345  10  15  20  25 

NUMBER  OF  ERRORS  CORRECTED  VS  PROJECT  TJmE 


man 

30  33  hour; 
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ANNEX  F 


ERPQR  CATEGORIES  AND  TYPES 


1.  Design  Errors 


The  fol 

wing  tyDes  of  errors  aooly  to  both  categories 

"System 

Oesign  Errors"  ana  "Program  Design  Errors": 

D 1 

Commun i c at i on  Error 

02 

Oesign  Negliqence 

DJ 

Forgotten  Cases  or  Steos 

Da 

Timing  Problems 

05 

Errors  in  1/0  Concents 

06 

Data  Design  Error 

07 

I n i t i a 1 i *a t i on  Error 

08 

Inadequate  Cheesing 

09 

Extreme  Conditions  Neglectea 

010 

Sequencing  Error 

Oil 

Indexing  Error 

012 

Looo  Control  Errors 

01  J 

Misuse  ol  boolean  Exoression 

0 1 a 

via  t hema  t i ca  1 Error 

015 

Reoresent at t on  Error 

016 

vi sunderst anci ng  o*  Proolem  Sc ec i f i c a t i 0 n s 

017 

Other  Design  Errors 

Coding  Errors 

Cl 

Mi  sunder s t andi nq  of  Design 

C2 

Neql igence 

C j 

1/0  Format  Error 

Ca 

uisolaced  Data  Declaration 

C5 

Multicle  Data  Declarations 

C6 

“issing  Data  Declaration 

C 7 

Inadequate  Data 

C8 

Initialization  Error 

C 9 

Error  in  Parameter  Passing 

CIO 

Inadequate  or  Forgotten  Checking 

Cl  1 

Level  Problems 

C 1 2 

Missing  Declarations  of  block  Limits 

Cl  J 

Case  selection  error 

Cl  a 

50  TO  Problems 

C15 

Comment  Error 

C 1 6 

Forgotten  Delimiter 

C 1 7 

Inconsistency  in  Naming 

CIS 

srong  Use  of  Nested  IF  Statements 

C 1 9 

Indexing  Error 

C20 

Inconsistent  Use  of  Variables  or  Data 

C2 1 

Sequencing  Error 

C22 

Flaq  usage  Problems 

C2J 

Syntax  Error 

C2a 

Looo  Control  Error 

C25 

Incorrect  Exit  from  Subroutines 

C26 

Languaae  Usage  Problems 

- 1Q9  - 


ANNE  X F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Foraotten  Statements 

CE8:  Reoresen t at i on  Error 

C2R:  Control  Seauence  Error 

C30:  Incorrect  Subroutine  Usage 

CJ1:  Other  Coding  Errors 


3 . Clerical  Errors 


a i : 

fjnual 

Error 

ae  : 

lenta 1 

Error 

J3  : 

Procedural  Errors 

a 4 : 

Other 

Clerical  Errors 

t*.  Debugging  Errors 

81  : I naoo roo r i a t e Use  of  Deougoing  Tools 

: Insufficient  or  I n aoo roor i a t e Selection 

of  Test  Cases  or  Test  Data 
83  : 'fisinteroretation  of  Deougging  Results 

Su  : Mi s i nt eroret at i on  of  Error  Source 

9b  : Negligence 

86  : Otner  Debugging  Errors 

S . Testing  Errors 

Tl  : Inaoeauate  Test  Case(s)  or  Test  Cata 

T2  : fi si nteroretat i on  of  Test  Results 

T3  ! Mi si nteroret at i on  of  Program  ioeci hcation 

T a : Meg  1 i gence 

T5  : Other  Testing  Errors 
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ANNEX  G Page  ! oT  5 

DIRECTED  G»aph  5e°P£SE’-TATtf!N 

PROJECT  3 : i 

Program  oart  : CORRECT  REACmAblLITY 


COMPLEXITY  MEASURES: 
NUMSE3  OP  STATEMENTS: 


1 0 


z ?■ 

C=  \CCES: 

4 

z* 

CP  A?C£  : 

4 

f : ) 

C n 

Cr  PATES  : 

2 

C iv 

ATIC  NUE6ER: 

v l G ) = 

2 V 

AEACHA3  X l_ I TT  Cr  NC0E3 : 


NO  C 3 
NCCE 
\CC3 
NCCE 


SL'M:  o«s.wy033 

A £ AC  l-A  3 ! L ! TT  IINCEX 
<-  c Mo:-rsr  OBASI-: 


1 . ECCOCO 


J 


AONE  x G 


Page  2 z*  % 


9 l °E C T £9  r, papr,  PEPPESt'*  T J T IQU 

PROJECT  s : 5 

Program  cart  : UPDATE  PE  AC  -*  AR  I L I T Y 


CO^PLExH^  P 1 3 u H E S : 


'•IUYBER  OF  STATEmE'IT3:  i 

<VUft6  Eft  CF  IV  C C c S : 2 

Mjft6ER  CF  A SC  S : L 

AU  ft 3 eft  CF  FATES:  1 

CYCLE ft ATIC  Al ft  3 E ft  : V ( G ) = L 


REACFAE  ILITY  CF  ftCOeS: 
ACCE  l : 1 


SLft:  2a»C30CJ 

S c #-C  ft  A 2 I L I TY  INCEX 
Cr  0 IREC  TEC  3 r A Fh  : 1.CC3300 


- 20?  - 
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ANNEX  H 


Page  1 


cf  3 


TEST  PHASE  DESCRIPTION 


Project  » S 3 

Test  run  # : 1 Including  1 Test  Steo 

Begin  ot  Test  ( day / t i me ) : 03/22/1000  End  of  Test  (day /tine)  : 03/22/1800 

CPU  time  for  necessary  comoiles  (in  sec.):  8.33 
a)8.33b)  c)  d)  e)  f)  g) 

CPU  time  for  TEST  run  (sec)  : 30. 7J 


Han  Hours  for  this  Test  run  : 8.0  (including  oreoarat'on  of 
tests) 


I ) 


TEST 

STEP 

C8JECTVE 

EXPECTED 

RESULT 

(TOLERANCE) 

ACTUAL : ERROR 
RESULT!  a 

1 

DA  r 
T I 

1 

Co^oar*  r?su 1 ts 

Program  oerfor- 

o 

A 

3/22 

o f pr*ev  i ous  test 

mance  should  not 

1000 

runs  with  results 

oe  c ranged  « i t n 

of  -noai  f i eg  vers- 

resoec t to  oath 

ion  of  the  orogran 

ana  lysis  as  oet  ng 

us i ng  sane  i nout 

soec i f i eo  in  oro- 

oat  a . 

ject  » 2. 

Additional  out” 

out  (such  as  cy- 

clomatic  numcer 

and  reac  hao i 1 i t y 

index)  must  ce 

yer i f i ed  Oy  desx 

c nec king. 
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ANNEX  H 


Project  a : 3 

Test  run  » : 2 Including  1 Test  Steo 

Begin  of  Test  (dav/time)  : 03/23/1800  End  of  Test  (day/time)  : 03/23/1800 

CPU  time  for  necessary  comoiles  (in  sec.):  7.90 

a)  7.90  b)  c)  d)  e)  f)  q) 

CPU  time  for  TEST  run  (sec)  : 139.00 


.'fan  Hours  for  this  Test  run  : 8.0  (including  oreoaration  of 

tests) 


TEST 

STE3 

08JECTVE 

EXPECTED 
9ESUL T 
(TOLERANCE) 

ACTUAL 

RESULT 

1 ) 

ERPOR 

# 

DAY 

TIME 

COMMENTS 

AND 

CODED  ERROR 
TYPES 

1 ) Record  »nen 

error  occurs. 

1 

Comoare  nana  cal- 

(n  c ases  of  di S" 

O.K. 

3/22 

culated  results  of 

creoanci«S  oro* 

1800 

NTOS  subrout i nes 

gram  shou 1 a o ro- 

o ♦ "oau  1 e 1 * i t * 

vide  tm  correct 

program  results. 

answer  otherwise 

Chec  k for  oi sc  re* 

results  nust 

oanc i es  (if  any ) . 

match.  In  case  of 

aiscreoancies  re- 

suits  must  oe 

veri fied  by  oes* 

3/23 

chec  x i ng . 
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ANNEX  M 
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TEST  PHASE  DESCRIPTION 


Project  a : 3 

Test  run  a : 3 Including  1 Test  Steo 

Beqin  of  Test  (day/time)  ! 03/23/2000  End  of  Test  (oay/ti 

CPU  tine  for  necessary  comoiles  (in  sec.):  7.87 

a)  7.87  O)  c)  d)  e>  f>  g) 

CPU  tine  for  TEST  run  (see)  : 171,7a 

Han  Hours  for  this  Test  run  : 3.)  (includina  oreoaration  o* 
tests) 


OBJECTvE 

1 ) 

EXPECTED  lACTUALIERPORlCAY 

RESULT  1 RESULT ! a ! TIME 

(TOLERANCE)  ! : : 

1 1 1 

Comoare  hand  cal* 

1 

1 

1 

1 

In  cases  of  dis-  ! 0 . * . 

3/23 

culated  results  of 

creoancies  oro-  ! 

2000 

NTDS  suoroutines 

gran  shou'C  oro-  ! 

o f nodu 1 e 2 with 

v i oe  the  co  r rec  t ! 

orogram  results. 

answer  otherwise  ! 

Chec  k for  disc  re* 

resul t s must  ! 

oanc i es  (if  any ) . 

match . In  case  of ! 

oi  sc reoane i es  re-! 

suits  must  be  ! 

veri f i ed  by  aesk  ! 

3/23 

checking.  ! 

2300 

me)  : 03/23/2300 


COMMENTS 
A NO 

CODED  ERROR 
TYPES 


1 ) Record  ««en 
error  occurs. 
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APPENDIX  D 
PROJECT  DESCRIPTION 

Project  * : 4 

Project  title:  DATA  RETRIEVAL  S Y S TEM 

Programmer  : HOFFMANN 
Programming  Language  : ALGOL 

Programming  environment:  I 3m/ 36 0 /6 7 , 0 3/ M V T > B A T C H 

and  Time  Sharing  (CP/CHS) 

Design  notes  : see  ANNEX  A 
Program  listing  : see  ANNEX  B 

Coding  notes  : see  ANNEX  C 

Debuaging  notes  : see  ANNEX  D 

Error  Listing  : see  ANNEX  E 

Final  statistics  : see  ANNEX  F 

Graphical  reoresent at i on  : see  ANNEX  G 

Test  chase  description:  see  ANNEX  H 

Starting  date:  IP  APRIL  77  Ending  date:  3 v A Y 77 

EXPERIMENT  DESCRIPTION 
1.  Project  description: 

A.  General  Description 

The  program  is  designed  for  usaae  under  CP/CMS.  It 
expects  an  input  file  laoeled  "DBASE  INPUT”  to  contain  data 
base  information  of  the  format  described  in  section  8. 

During  i n i t i a 1 i z a t i on  all  data  base  information  is  read 


into  program  memory.  After  this  the  user  may  operate  upon 
the  data  using  functions  as  described  in  section  C.  All 
functions  are  input  via  terminal. 

Most  of  the  terminology  being  used  throughout  the  pro- 
ject is  non  standard.  Therefore  the  following  explanation 
is  provided  for  better  understanding: 

a)  DATA  BASE  MEMBER:  see  description  under  B.  1. 
n)  ATTRIBUTE  VALUE  PAIR:  see  description  under  B.  2. 

c)  MEMBER  ID:  The  program  assigns  a MEMBER  ID  to  each 
data  base  memoer  depending  on  the  input  sequence. 
MEMBER  10s  are  integers  (1#2»3  ...).  The  MEMBER  ID  can- 
not be  changed  by  the  user#  however,  it  may  be  used  to 
reference  any  particular  data  Case  memoer. 

d)  ATTRIBUTE  ID:  The  program  assigns  an  ATTRIBUTE  ID 
to  eacn  new  ana  distinct  ATTRIBUTE  NAME  being  entered 
during  the  initial  incut  of  the  data  base  members. 
ATTRIBUTE  IDs  are  integers  (1,2,  ...64)  depending  also  on 
the  inout  seauence.  Each  ATTRIBUTE  ID  is  associated 
which  a uni  due  ATTRIBUTE  NAME  and  provides  a way  of 
referencing  any  particular  ATTRIBUTE  NAME  (see  section  C 
"LISTA" ) . 

e)  ATTRIBUTE  NAME:  An  ATTRIBUTE  NAVE  is  a character 
String  of  lenath  1-64  orecedinq  the  ":M  in  the  input  of 
an  ATTRIBUTE  VALUE  PAIR  (see  B.  2.). 

f)  ATTRIBUTE  VALUE:  An  ATTRIBUTE  VALUE  is  a character 
string  following  the  in  the  definition  of  an 
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ATTRIBUTE  VALE  PAIR  (see  B.  2.). 

g)  KEY  ATTRIBUTE:  Tne  KEY  ATTRIBUTE  refers  to  the 
second  distinct  attribute  name  being  input.  (The  second 
attribute  name  has  been  chosen  because  the  main  usage  of 
the  proqram  will  have  data  base  members  which  are  iden- 
tified uniquely  by  their  second  attribute  value  pair.) 
Therefore  the  ID  of  the  KEY  ATTRIBUTE  is  initially  2. 
however,  this  may  oe  changed  by  the  user  to  any  other 
ATTRIBUTE  ID  by  usinq  the  function  "KEY"  (see  section 
C)  . 

The  KEY  ATTRIBUTE  is  not  always  the  "key  attribute"  in 
the  sense  being  used  in  most  data  bases,  because  it  does 
not  necessarily  allow  a uniaue  identification  of  all 
data  base  members.  (Instead  the  MEMBER  ID  can  be  viewed 
as  internal  key  which  always  allows  a uniaue  identifica- 
tion of  each  individual  data  base  member.) 

h)  CONTROL:  If  a data  base  member  is  taken  into  con- 
trol by  using  the  function  "CONTROL"  an  internal  refer- 
ence is  set  by  t^e  program  which  allows  subsequent  exam- 
ination of  ATTRIBUTE  VALUE  PAIRS  of  this  particular 
member  by  usinq  functions  "FA"  and  "ATTR". 
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B . InDut  Format  : 

1.  Description  of  a Data  Base  Member: 

A data  base  member  is  defined  by  1 - 6U  different 
attribute  value  pairs  followed  by  the  termination  symbol 
" # " . After  the  last  data  base  member  an  additional  ter- 
mination symbol  has  to  be  inserted. 

2.  Description  of  Attribute  Value  Pairs: 

Each  attribute  value  pair  has  the  following  format: 

<attribute  nam?>:<attribute  v a 1 u e > a 
where  "attrioute  name"  is  a character  string  of  length  l 
- 6 3 not  including  ana  "attribute  value"  is  one  of 

the  following  types: 

a)  <single  discrete  value> 

A single  discrete  value  is  a character  string  of 
length  0 - n,  where  n is  limited  Dy  the  amount 

of  memory  available  (see  section  0). 

A character  strina  of  length  0 will  be  reel acea 
internally  bv  "***"  and  specifies  an  unknown 
value. 

b)  <$<mu)tiple  discrete  values>> 

Multiple  discrete  values  are  any  number  of  sin- 
gle discrete  values  separated  by  " . The  total 
string  length  of  all  values  including  separating 
may  not  exceed  6a. 


2 1 0 


c)  <3<range  value>> 

A range  value  is  a string  of  length  6 - 64  of 

the  format : 

<lower  limit>  TO  <uoper  !imit> 
where  "lower  limit"  and  "upper  limit"  are  in- 
teger values  and  <lower  limit>  <=  <upper  limit>. 

Examol es: 

a)  Attribute  value  pair  with  single  discrete 
value: 

NAME : SMI TH# 

b)  Attribute  value  pair  with  multiple  dicrete 
values: 

CHECKING  ACCOUNT: $604 020 >50 033# 

c)  Attribute  value  pair  with  ranae  value: 

MONTHLY  PAYMENTS:  <555  0 TO  725# 

3.  Number  of  data  base  members: 

The  data  base  may  contain  1 -150  members  depending 
upon  the  amount  of  memory  being  used  for  the  storage  of 
values  (see  section  0). 

4.  Termination  of  input: 

In  order  to  terminate  the  input  of  data  base  infor- 
mation an  additional  termination  symbol  ("#")  has  to  be 
inserted  after  the  the  termination  symbol  of  the  last 


data  base  member 


5.  Reserved  characters: 

All  characters  other  than  M : " , " , " , " # " , and 

"•i"  may  be  used  without  restriction. 

6 . Samp 1 e i nout  : 

NAME  : ME YER  « FIRST  NAME:J0E«  AGE : 57«S  NAMEjSMITH# 
FIRST  NAME:mIKE#  AGE : a a NAME : NEWMAN# 

FIRST  NAMEsMARY  ANNS  AGE:  laasu 

NOTE: 

a)  between  attribute  value  oairs  any  number  of  blank 
characters  (0,1,2#...)  will  be  ianored  by  the  program. 

b)  Any  number  of  blank  characters  preceding  a value  will 
be  ignored.  (In  the  example  the  age  of  the  last  data 
base  member  will  be  stored  as  "3U".) 

c)  Any  number  of  blank  characters  greater  than  one 
within  a value  or  attribute  name  will  be  reauced  to  a 
single  blank.  (In  the  example  the  first  name  of  the  last 
data  base  memper  will  be  stored  as  "MARY  ANN "and  t n e 
attribute  name  of  the  second  member  wilt  be  stored  as 
"FIRST  NAME"  although  it  was  incut  as  "FIRST  NAME"  .) 
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C.  Function  Commands: 


1.  Functions  without  input  oarameters 


INPUT  j 
LIST  A ! 


LISTC 


LISTD0ASE 


1 

LISTALL  i 


+ 

SWITCH  ! 


♦ 


FUNCTION  DESCRIPTION 


Ail  attribute  names  are  listed  together  with  their 
ATTRIBUTE  ID.  (For  usaqe  of  ATTRIBUTE  ID  see 
functions  " A T T R ” and  "KEY".) 


All  available  commands  are  listed. 


All  data  base  members  are  listed  orintinq  all  of 
their  attribute  value  oai rs  together  with  MEMBER 
ID  of  each  data  base  member.  (For  usage  of  MEyBER 
ID  see  functions  "CONTROL"  and  "LIST!*".) 


All  data  base  members  are  listed  printing  only 
their  KEY  ATTRIBUTE  and  its  value  preceded  by  the 
MEMBER  ID.  (For  usage  of  MEMBER  ID  see  functions 
"CONTROL"  and  "LISTM"  . The  KEY  ATTRIBUTE  is  the 
second  attribute  of  the  first  data  case  member 
by  default  and  may  be  chanoed  bv  using  the 
function  "KEY".) 


This  allows  the  user  to  direct  the  subseauent 
output  from  terminal  to  a file  labeled  "DBASE 
OUTPUT"  or  or  vice  versa.  To  change  the  output 
from  terminal  to  file  might  be  appropriate  if 
large  output  is  expected.  The  contents  of  the 
output  file  may  be  obtained  using  the  CMS  command 
"OFFLINE  PRINTCC  DBASE  OUTPUT". 


Termination  of  the  program 


2.  Functions  which  reauire  only  one  input  Parameter 


INPUT  OF 
FUNCTION 

PARAMETER 

FUNCTION  DESCRIPTION 

L I S T M 

<mem0ER  ID> 
must  be  i nteger 

All  attribute  value  oairs  of  data  base 
member  soecified  by  MEMBER  ID  are  listen. 

CONTROL 

<MEMBER  I0> 
must  be  i n t eae  r 

Program  takes  data  oase  memter  soecified 
by  MEMBER  ID  into  control.  (For  usage  of 
this  function  see  functions  "FA"  ana 
" ATTR"  . ) 

AT  TR 

<ATTRIBUTE  I D > 
must  be  integer 

List  value  of  attribute  soecified  by 
ATTRIBUTE  ID  of  data  base  memoer  in 
control.  (To  select  orooer  ATTRIBUTE  ID 
see  function  "LISTA".  To  take  a member 
into  control  see  function  "CONTROL".) 

FA 

ATTRIBUTE 

N A V £ > 

must  be  a Cha- 
racter string 
of  length  1 -b 3 
followed  by  " : " 

List  value  of  attribute  soecified  by 
ATTRIBUTE  NA^E  of  member  in  control. 

(To  take  a member  into  conto'  see 
function  "CONTROL".  To  select  an  existing 
ATTRIBUTE  NAME  see  function  "LISTA".) 

*EY 

ATTRIBUTE  ID> 
must  be  i nteqer 

Change  key  attribute  of  data  base  to 
attibute  soecified  by  ATTRIBUTE  ID. 

(For  usaae  of  key  attribute  see  functions 
"LISTALL"  and  "FIND  ID".  To  select  an 
ATTRIBUTE  ID  see  fur^tion  "LISTA".) 

N o t e : 

I f 

oa  r ame  t e r 

i s r eau i red  to  oe 

an  inteqer 

an  v 

c t r 

i nout 

w i 1 

11  force 

the  oroaram  t o 

terminate. 

Other 

w r eng 

i nout 

will 

1 cause  an 

aDDrooriate  error 

me s sage . 
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3.  FIND  and  FIND  ID: 

a)  Function  Descrition: 

These  two  functions  allow  the  user  to  find  a'l  data  case 
members  which  satisfy  1 thru  4 conditions  soec i f i ea  by 
subseauent  input.  (By  a condition  is  meant  an  attribute 
value  oair.  A member  satisfies  the  condition  only  if  it 
owns  this  particular  attribute  value  pair  or  if  it  owns 
the  attribute  without  a value  specified.  The  latter 
case  will  be  indicated  by  printing  "***»"  as  attribute 
value  in  the  outout  following  a Query. ) After  the  condi- 
tions have  been  inout  the  program  tests  all  data  case 
memcers  as  to  whether  tney  satisfy  all  conditions.  Only 
those  members  which  satisfy  all  conditions  of  the  auery 
are  listed.  Using  "FIND"  all  of  their  attribute  value 
pairs  are  printed  whereas  using  "FIND  ID"  only  the  key 
attribute  and  its  value  is  printed.  (To  change  the  key 
attribute  of  the  data  base  see  function  "KEY".)  If  no 
conditions  are  entered  all  data  case  members  will  ce 
1 i st  eo . 


b)  Input  of  Conditions: 


CONDITIONS  1-4: 


Input  of  <ATTRIBUTE  NAME>  ! 

- - - - 

ATTRIBUTE  NAME  can  be  any  i 

string  of  length  1-63  1 

followed  by  (it  is  ! 

necessary  that  the  1 

ATTRIBUTE  NAME  matches  with; 

one  of  the  existing  ! 

ATTRIBUTE  NA^ES)  * 

If  an  ATTRIBUTE  NA^E  cannot! 
be  found  an  error  message  ! 
is  printed  ana  the  user  can! 
reoeat  the  i nput  . ! 


Input  of  <attribute  value> 


within  Queries  attribute  values  are 
restricted  to  oe  l-o4  characters.  (If 
the  value  stored  is  longer  than  o4 
characters  the  comparison  is  only 
carried  out  uo  to  the  last  character 
of  the  auerv  value.  ) 

Since  all  values  are  stored  without 
preceding  blank  characters  the  auery 
value  should  not  have  blank  characters 
in  front  of  the  value  to  be  tested  for. 


c)  Termination  of  Query  Input: 


User  i nput  s : 

Q 

d )  Samp  1 e 
FIND 
NAME  : 
SMITH 
AGE: 


Input  for  FIND  and  FIND  ID: 


?15 


22 

Q 


FIND  ID 
AGE: 

1 0« 

Q 

NOTE: 

1.  All  incut  for  all  functions  must  not  be  preceded  by 
blank  characters. 

2.  Each  incut  has  to  be  on  a separate  line.  The  incut  is 
entered  by  hitting  the  carriage  return  key. 

D.  Limits  of  the  Program: 

The  total  number  of  different  attribute  names  may  not 
exceed  o4.  Assuming  100  data  base  members  having  04  attri- 
butes each,  the  overhead  used  for  implementation  (o5  items 
for  eacn  data  base  member)  would  reduce  the  amount  allocat- 
able  items  of  M2  (20,000)  to  13,500  items.  Each  item  of  M2 
can  be  used  to  store  1-4  characters.  The  number  of  remain- 
ing items  divided  by  the  number  of  data  base  members  gives 
the  average  number  of  items  available  for  each  data  base 
member  ( approximately  135).  Thus  the  total  amount  of 
remaining  storage  locations  allow  approximately  54,000  cnar- 
acters  for  values  which  means  that  the  average  string  lenqtn 
of  values  will  be  less  than  B.  The  program  assumes  that  at 
least  two  different  ATTRIBUTE  NAMEs  are  entered.  If  this  is 
not  true  the  user  has  to  chanae  the  KEY  ATTRIBUTE  to  "1". 

E.  Error  Messages: 

All  error  messages  are  self  explanatory,  Error  al  and 
s<2  will  cause  the  orogram  to  terminate.  This  will  be  trie 
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case  when  alt  memory  locations  are  exhausted  (error  *<1)  dur- 
ing the  input  of  data  base  information  or  more  than  1000 
memoers  are  soec i f i ed  (error  n 2). 

Error  J*3  indicates  that  an  attribute  name  is  lonqer  than 
63  characters.  (In  this  case  the  problem  might  be  solved  Dy 
examining  the  input  file.) 
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2.  Programmer's  background: 

a)  Experience  in  programming: 


Oct 

1970  - May 

1971 

Programming  courses 

May 

1971  - April 

1972 

Module  Programmer 

May 

1972  • June 

1974 

Work  in  Test  and  Simulation  Depart 

ment  at  the 

NAVAL  COMMAND  and  CONTROL  SYSTEMS 
COMMAND  (FEDERAL  GERMAN  NAVY) 

Testing  of  tactical  real  time  systems 
March  19  75  - Jan  1977  Student  at  the  NAVAL  POSTGRADUATE 

SCHOOL,  Monterey,  Computer  Science 

b)  Experience  in  testing: 

Two  years  of  work  in  testing  ana  simulation. 

c)  Experience  in  the  area  of  the  given  problem: 

Usage  of  similar  oata  structures  in  previous  programming 
projects. 

d)  Experience  in  the  progamminq  1 anguage  being  used: 
Experience  over  a period  of  18  months  in  more  than  10 

programming  projects.  (Total  number  of  source  statements 
produced  during  that  time  „as  more  5000.) 

3.  Psychological  factors: 


a)  Did  the  programmer  like  the  project? 


Yes 


b)  How  aoes  the  orogrammer  like  the  orogramminq 


1 aoquage? 

Favorite  programming  1 anauage. 

c ) Was  the  programmer  satisfied  by  the  way  the  problem 
was  spec i f i ed? 

Only  minor  criticism. 

d)  How  did  the  programmer  like  the  Drooramminq  environ- 
ment ? 

The  facilities  (study  room,  card  ounch  room)  were  not 
conducive  to  efficient  orogramming  because  of  restricted 
space,  cad  lighting  and  noise. 

s)  Other  factors! 

The  recording  of  the  experiment's  data  during  the  cro- 
ject  affected  soeed  and  concentration  consideracly. 
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u.  Comments  on  Documentation 

For  the  documentation  of  each  software  development  chase  a 
special  documentation  form  has  been  developed.  These  forms 
are  designed  to  provice  a firm  guideline  for  the  experiment 
programmer  to  recoro  all  data  of  interest  for  sutseauent 
error  analysis. 

- begin  and  end  of  each  steo  was  recoroec  with  respect  to 
day  and  time. 

- Each  error  was  recorded  when  it  is  discovered.  error 
was  tnen  identified  by  a uni oue  error  number  (1»2»...). 
Furthermore  the  time  of  discovery  and  the  error  type  (using 
error  types  listed  in  ANNEX  F)  were  recorded. 

“ If  appropriate#  comments  about  error  discovery#  reason  why 
the  error  was  made#  etc.  were  documented  in  ANNEX  E2. 

- For  each  error  the  phase  in  which  the  error  was  mace#  the 
phase  in  which  the  error  was  discovered  and  the  time  scent 
to  correct  the  error  was  recorded  in  ANNEX  El. 

- For  each  steo  in  any  one  of  the  software  development 
chases  the  day/time  of  begin  and  eng  was  recorded.  In  addi- 
tion# the  time  (in  man  hours)  for  each  steo  was  recorded. 
This  excludes  the  overhead  used  for  documentation  of  the 
e xoe  r i men  t da  t a . 
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ANNEX  A 


Page  1 of  3 


r 


WORKSHEET  FOR  OESIGN  PHASE  AND  DESIGN  RE V I Ert  PHASE  OF  PROJECT  a a 


M A h 

STEP 

problem  and  planned  solution 

ALTERNATE 

DAT 

hours 

ERROR 

COMMENT 

9 

solutions 

TIME 

/STEP 

u 

1 

Analysis  o*  reau  i r e-nen  t s of 

4/1  0 

oro j ec  t • 

•Since  almost  all  exsected 

Storage  in  fixed 

uoo 

S.O 

values  are  of  li f ferent  lenqth 

1 enot h s s t r i ng 

usage  of  linked  lists  for  sto- 

arrays  wou 1 d 

raae  of  all  attribute  values 

-aste  too  much 

seems  to  be  aoorooriate. 

-Study  of  I/O  caoaoilities  un- 
der CP/CMS. 

memory. 

-View  of  data  case  memoers: 
Eacn  data  oase  member  ^as  a 
uni Oue  identifier  (memoer  3). 
Pgr  each  mei'cer  ma*,  oJ  di*4e- 

a)  Let  "u^oer  of 

rent  attribute  value  oairs  Can 

different  attri- 

be  defined  bv  inouf.  Maxi-un 

butes  be  an  in- 

String  ■enat"'  ‘or  anv  attri- 

out  oarameter. 

bute  is  e>4.  The  total  number 

o)  Use  a 'jsh 

Of  different  attributes  in  the 

tac'e  to  store 

data  oase  mav  also  not  exceed 

attrioutes.  This 

64  (aroitrarily  chosen  limit). 

«ou Id  max e 
S t r i nq  1 enqt  h s 
of  attributes 
inaecencent  of 
the  array  size 

4/10 

of  the  attrioute 
table. 

190  0 

2 

Oefine  gate  itfucturss: 

4/1  0 

9.0 

-Attributes  are  identified  ov 
a character  string  of  lenath 
64  and  stored  in  a string 
array  (ATTRIBUTE). 

Each  attribute  can  oe  uniquely 
identified  by  its  oosition 
within  tKe  attrioute  array 

1900 

Primitives 

(ATTRIBUTE  ID). 

for  :at  a 

-Values  o v attributes  are 

s t rue  t u res 

stored  in  a linued  list  strut- 

could  oe 

ture  using  a 6IT5  ARRAY  (^2). 

coo i edf  re- 

(Length  of  M2  (124,000  bytes) 

4/10 

cro j ec  t 

limits  the  amount  of  informa- 
tion to  Oe  stored  in  the  data 

Hase . ) 

0900 

»2. 
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Remarks : 


a ) 

Li nked  1 i st 

of  oat  a base 

'nep'be  rs : 

i 

• 

! 

t 

1 

• i •••*■  i 

i a: 

_ i ..... i 

i 

i 

ct  r lot  r---> 

! * * ! * 

i 0 1 P - • • - 

• i • i 

• > 

attribute  value  attribute  value 

list  of  member  1 list  of  member  2 


n J 

J pt  r 

• i .. 

NIL  i 

V 

attribute  value 
list  of  membe  r n 
(last  memoe r ) 


o)  Structure  of  attribute  value  lists: 


A 1 1 

A2I 

A ( n J 

Dt  r 

i .. 

ot  r - -- 

Ct  r 

o t • • • 

ot  r 

NIL 

value  structure  value  structure  value  structure 

of  first  attrioute  of  secona  attribute  o‘  it"  attricute 

.Were  At,  A 2,...  recre sent  tie  ATTRIBUTE  IDs  of  corresoonamg 
v a I ues 

c)  Value  structure: 


t lotr 

t ! nil 

value  repre: 

SENTATION. . I 

ah«r(  t r ec  resent  3 t"e  tyoe  of  tne  value: 
t so : s i "Q ' e value 
t = 1 : range  values 
t -2'  »u I t i b I e values 

* is  usea  as  oelimiter. 

Usaoe  of  for  value  reo resen t at i o"  indicates  tnat  value 

is  not  known. 
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WORKSHEET  FOP  CESIGN  PHASE  AND  DESIGN  REVIEW  PHASE  OF  PROJECT  a 4 


PROBLEM  and  planned  solution 


ALTERNATE 

SOLUTIONS 


! ham  : 

DAY  !HOURS!E°PPR 

TIMEi/STEP!  » 


3 !a)  Define  utilities  for  incut  ! 'a/20 

lof  data  base  i n format i on : ! i 0900 

: -FILL, EXTRACT, GETCHAR, SKIP  j ! 

1 -GET  OLD  FILE  INFORMATION  1 I 

! -STORE, CLEAR  IN, CLEAROUT 

! -CARDIN, PRINTOUT  ! 

:-typin,typout  : : 

1-FILE  I 0, TERMINAL  10!  ! 

l-GET  ATTRIBUTES  AND  VALUES  1 1 

1-GET  ATTRIBUTE, GET  VALUE  1 1 

i-FIND  ATTRIBUTE  1 1 

10)  other  uti 1 itiesi  1 1 

1 -SPACE, ERROR, DIAGNOSTICS  1 lu/20 

I-INITIALIZE  all  1 11300 

a la)  Define  coiwanfl  c aoab i I i t i es 1 14/21 

1-List  all  attributes  (LISTA)  J !0°00 

1-List  all  me-cers  (using  only  1 1 

lone  attribute  value  oair  1 1 

Icnosenbyuser)  (LISTALL)  1 1 

1-List  all  ne’Oers  (orintino  1 1 

1 a 1 1 attribute  value  cairs  of  1 1 

leacb  member  (LISTDBASE)  1 1 

1-List  all  available  commands  1 1 

1 (LISTC)  1 1 

1-List  all  attribute  value  ! 1 

loairs  of  a single  "enter  1 1 

1 (LISTM)  | | 

1-Select  a single  "enter  for  ( 1 

Iretrieval  of  oarticula'  values!  1 

lof  interest  (CONTROL)  1 1 

1-List  a oarticular  attribute  ! ! 

lvalue  oair  of  "*«oer  in  con-  ! 1 

Itrol  (FA.ATTR)  1 1 

1-rino  all  oata  base  "e*oers  1 i 

luhich  »ith  given  conditions  1 1 

1 (FIND,  FIND  10)  1 1 

i-Switcn  outout  fro"  terminal  1 1 

Ito  file  and  vice  ver sa ( Sw I TCH ) 1 1 

1-Cbanqe  kev  attribute  (KEY)  1 1 

l-0btain  diagnostics  (0)  1 ! 

1-Sucer  user  mode  (Su)  1 1 

lb)  All  subroutines  which  han-  1 14/21 

Idle  commanos  are  named  accor-  1 11100 

1 di ng I V . 1 1 

Remarks!  The  last  t»o  functions  are  not  reoui red,  however 
effectively  used  for  testing  ourooses.  They  also  suoeort 
eitensions  of  t"e  orogram. 
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WORKSHEET  FOR  OESJGN  PHASE  AND  DESIGN  REVlErt  PHASE  OF  PROJECT  s a 


! 

; man  ; 

STEP 

PROBLEM  AND  PLANNED  SOLUTION 

ALTERNATE 

DAY  {HOURS 

ERROR 

COMMENT 

0 

SOLUTIONS 

T Iu£ ! /STEP 

0 

5 

Analysis  0*  i mo  1 emen t i ng  FIND 

• 

4/221 

(FIND  ID  ) Tunc  t i on . 

1230!  1.5 

9 e H niton  of  suorout ines: 

i 

1 

-GET  CONDITIONS 

• 

-GET  REOUEST 

-GET  Q VALuE  (functions  wfii  e A 

• 

allow  user  co  AeHnj  dueries) 

• 

-TEST  CONDITIONS  (evaluation 

1 

1 

0 ♦ auer i es ) 

• 

Define  functions  to  test  foe 

• 

f 

single  discrete  values: 

• 

-finoattrisute 

i 

t 

-E'lUAL, MATCHING 

• 

-FI  NO  «E«8ER 

1 

Define  functions  to  orint  re- 

1 

• 

suits: 

1 

• 

-eRITE  RESULTS 

• 

1 

-LlSTATTRI9uT£(value) 

• 

1 

All  -ne-oers  «nicn  satisfy  user 

1 

defined  conditions  are  first 

t 

stored  witnin  a linked  list. 

» 

The  result  of  all  eonoarisions 

l 

• 

are  orintefl  4f ter  all  data 

<a  / 2 2 ! 

mercers  have  oeen  e*amined* 

14301 

1 

<5 

Design  of  retrieval  functions 

4/23  1 

for  multiple  values  and  ranqe 

1530!  o.O 

values: 

-COMPARE  MANY 

1 

-LLIMIT,ULIMIT 

4/25  1 

(return  uooer  and  lower  limit) 

1100  1 
• 

7 

Design  review  and  inole^erta- 

4/271 

t i on  of  i -o rovemen t s . 

13001  1.5 

4/271 

2030  1 
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annex  C Page  I o f 2 

WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  « : 4 
Beginning  of  Coding  (aay/time)  s 4/20/1400 
End  of  Cooing  (day/time)  : 5/02/1200 
fan  hours  S 24.5  (including  ounching  of  cards) 


1 ) 


COOING 

BEGIN  ! EN0 

0AY/TIHE  ! day/time 

program  PART 

EROQP ; Day 
« {TlfE 
1 

COMMENT 

04/20/1400 

04/20/ 1 BOO 

Cod  i no  o f or  i «n  > - 
t i ves  and  ut i H - 
t i es 

1)  PfCord  *ren  ar^or  »s 
detected. 

A great  mjfrCer  of 
oridii  t i ves  could  c e 
cooied  frow  croject  a£. 

04/20/2030 

04/20/2330 

Punch  » nq  cares 

04/2 ! / 1 330 

04/21/ 1530 

Codina  of  cc*"* 
wand  f unc  t i on  s 

04/21/1530 

04/21/1830 

Punching  cans 

b 

1330 

CR 

04/21/2030 

04/21/2130 

Coding  Of  CCn~ 

m a n d functions 
ana  ouncri nq 
Caras 

04/21/2230 

04/21/2330 

Coding  of  CO»- 
wand  f unc  t i ons 
anq  punc b i no 
cards 

04/22/1500 

04/22/1800 

Cod i na  of 
retrieval 
f unc  t i ons 

04/22/1*30 

04/22/ 1R30 

Punch i nq  cares 

04/25/1 330 

04/25/ 1630 

Coding  of  re- 
retrjeval 
funct i ons  for 
multiole  values 
an 0 range  values 

255 


ANNEX  C 


Page  2 of  2 


WORKSHEET  FOR  COOING  PHASE  OF  PROJECT  « : u 
Beginning  of  Coding  (dav/time)  : u/20/1400 
End  of  Coding  (dav/time)  : 5/02/1200 
Han  Hours  : 2U.5  (including  Punching  of  cards) 


1 ) 


COOING 

BEGIN  ! END 

0AY/TI«E  : 0 A ' / ’ THE 

PROGRAM  PART  1 ESROP  ! DA  Y 

: * stive 

COGENT 

• 

• 

1 

• 

1)  ^ecorg  wr»e^  error  is 

1 

1 

1 

• 

le*.  ec  t eo . 

OU/25/2 1001 

Punch i na  cares  ! 

1 

• 

: 35 

2100 

A 1 

» 

1 

! 3o 

2200 

A 1 

1 0U/25/2200 

1 

1 

1 

0R/26/ 1 5 SO  ! 

Coding  of  ! 

• 

1 

aeougging  aios  J 

l 

• 

ang  ounemng  • 

i Oa/20/  laJO 

cards  J 

Ju/27/2030  ! 

• 

Changes  accor-  ! 

1 

1 

d i nq  t c aes i gn  i U8 

2050 

C 1 7 

! 04/27/2100 

review  ; 

05/02/ 1100! 

Coflinq  of  new  | 

1 

• 

procedures  to  ! 

Change  due  to  error  a 50. 

1 

1 

handle  range  ! 

• 

1 

value  eono a r i son ! 

1 

1 

ang  ounchinq  J 

! 05/02/ 1200 
• 
t 

carqs  J 

i 

i 
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WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  * : a DEBUG  Run  a : 1 

Begin  of  DeOug  Run  (aay/time)  t 00/20/2330 
End  of  Debug  Run  (day/time)  : 00/21/2030 

a of  Debug  Steps  incl.  in  Debug  Run:  a CPU  time  for  Deoug  run  (sec):  0.0 

CPU  time  for  necessary  compiles  (see)  : 1R.3R 

a)  3.03  b)  a.a2  c ) 5.3a  o)  5.70  e)  f)  d) 

'fan  Hours  for  this  Debug  Run  : 2.S  (including  preparation  of  debug  run) 

man  1) 


STEP 

a 


PROGRAM 

PART 


P r i m i ■ 
t i yes 
ano 

utili- 

ties 


Primi- 
tives, 
utili- 
ties, 
i nou  t 
f unc t - 
i ons 


OBJECTIVE  ANO  EXPECTED 
RESULT 


ACTUAL 

RESULT 


DAT ! HOURS  1 ERROR 
TIME! /STEP!  a 


Get  error  free  co-oile 


Get  error  free  co»oi le 


reoeat  step  2 


repeat  steo  2 


5 c omo  i 1 e 
errors 


1 c omo i 1 e 
error 


1 c omo i 1 e 
error 


O.K. 


1 1 ) Record 

a/20 

! error 

2330 

1 .0 

! CCCurs 

a/21 

i 

• 

1030 

i :co 

2 ; ai 

3 ! A l 

a/21 

a : A l 

1100 

5 : a i 

a/21 

i 

i 

t 

1 b 30 

0.5 

i 

« 

1 R00 

7 C 1 7 

a/2  l 

1 

1 

1 

1 R 30 

1 

a/2  1 

I 

< 

1 

1 R 30 

0.5 

• 

2000 

S ! C 1 2 

a / 2 1 

i 

2000 

1 

a/2l 

2000 

0.5 

1 

• 

a/21 

• 

• 

2030 

• 

I 

• 

COMMENTS 
ANO  COOED 
ERROR  TYPES 
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WORKSHEET  POR  DEBUGGING  PHASE 
PROJECT  a : 4 DEBUG  Run  # : 2 

Begin  of  Debug  Run  (day/ tine)  : 04/21/2130 

End  of  Debuo  Run  (day/tine)  : 04/22/1830 

a of  Debug  Steps  i nc 1 • in  Debug  Run!  6 CPU  tine  for  Debug  run  (seel:  IP. 83 
CPU  tine  (or  necessary  conoiles  (sec)  : 39.55 
a)  8.48  b)  5.52  c)  5.08  c)  «. 97  e)  8.20  f)  9.m  a) 


''an  Hours  for  this  OeouQ  3un  : 5.5  (including  oreoaraticn  of  debug  run) 

VAN  I ) 


STEP : PROGRAM 
* ; PART 
• 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL 

RESULT 

DAY  ; HOURS ; ERROR 
T TME  j /STE3!  a 
• 

COMMENTS 
AND  COOED 
ERROR  TYPES 

• 

• 

1 ) Rec ora  when 

1 i A II 

Get  error  free  co^oile 

2 z ono i l e 

a/21 

error 

I exceot 

error s 

2130 

0.5 

occurs 

! retrie- 

2130 

0 

C 7 

: val 

1 0 

A 1 

! f unc  t - 

4/21 

i ions 

2200 

2 ! 

reoeat  steo  1 and  check 

1 o rear  an 

4/21 

1 

) 

initial  i zaMon 

error 

2200 

0.5 

• 

2200 

1 1 

07 

1 

1 

4/21 

1 

1 

2230 

3 : 

reoeat  steo  2 

5 orogran 

4/2  1 

• 

i 

errors 

2330 

2.5 

• 

4/22 

i 

i 

10  00 

12 

C28 

t 

t 

1020 

13 

C2  1 

i 

i 

1030 

14 

C2  1 

i 

« 

1 100 

15 

C2« 

i 

• 

1110 

1 0 

012 

t 

• 

4/22 

i 

i 

1200 

a -Ail 

Get  error  free  compile 

2 c ono  i 1 e 

4,22 

! exceot 

(including  command 

errors 

1200 

0.5 

} r e t r i e - 

f unc  t i ons ) 

1200 

1 T 

Co 

; val 

1230 

18 

C 1 7 

! f unc  t * 

4/22 

J i ons 

1230 

5 ! 

repeat  step  A and  check 

2 program 

4/22 

I 

initialitation 

errors 

! 400 

1 . 0 

• 

1 4 JO 

19 

012 

1 

4/22 

* 

• 

1500 

20 

C 2 7 

5 } 

reoeat  steo  5 

0 . K • 

4/22 

• 

i 

1800 

0.5 

k.  * 
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annex  0 Page  1 of  10 

worksheet  for  debugging  phase 

PROJECT  # : 4 DEBUG  Run  « : 3 

Begin  of  Debug  Run  (aay/time)  5 04/22/1330' 

End  of  Debug  Run  (day/tine)  : 04/22/2200 

» of  Debug  Steps  incl.  in  Debug  Run:  2 CPU  tine  for  Debug  run  (sec):  11. 03 

CPU  tine  for  necessary  conoiles  (sec)  : 17. 75 

a)  3.71  b)  9.04  c)  o)  e)  f)  d) 

Man  Hours  for  tnis  Debug  Pun  : 2.5  (including  oreoaration  of  denud  run) 

MAly  1 ) 


ste° 

PROGRAM 

OBJECTIVE  AMD  EXPECTED 

ACTUAL 

day  ; HOURS ! ERROR 

C0*WF\TS 

9 

° A R I 

RESULT 

RESULT 

TIME ! /STE= 

3 

AMO  CODED 

ERROR  TYPES 

1 ) Record  »nen 

error 

occurs 

l 

A 1 1 

Get  error  free  comcile 

"5  c o m o i 1 e 

J/22 

ex cect 

errors 

1930 

0.5 

f unc  t - 

1945 

21 

A 1 

tions 

1950 

22 

42 

to  com- 

1 q55 

23 

C 2 3 

oa  re 
range 
values 
or  mu  1 • 
tide 
dis- 
crete 

4/22 

values 

2000 

2 

reseat  stec  1 and  eia»ine 

4/22 

storage  of  data  ease  in- 

format  ion 

O.K. 

2000 

(all  attributes  and  values 
should  ce  imolementea 

4/22 

according  to  design) 

2200 

- 259  - 
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hQRKShEET  FOR  DEBUGGING  PHASE 
PROJECT  * S 4 0E6UG  Run  » : 4 

Begin  of  DeOug  Run  (day/time)  : 04/22/2200 

End  of  Debug  Run  (day/time)  : 04/2S/1100 

» of  Debug  Steps  inel.  in  Debug  Pun:  6 CPU  tine  for  Debug  run  (sec):  43.47 
CPU  tine  for  necessary  comoites  (sec)  : 73.14 
a)  11. 3o  b)  9.73  c)  9.84  o)  9.4©  e)l0.87  f)10.49  g)  10.39 


Man  hours  for  this  Debug  Run  : 7,5  (including  oreoaration  of  oeoug  run) 

MAN  1 ) 

stepiprogram:  objective  ano  expected  : actual  : dayihoursierrcr!  com 

s : PART  ! RESULT  ! RESULT  ! T I E 1 / S T E P • a AND 


COMMENTS 
ANO  CODED 
ERROR  TYPES 


A1  1 

Chec«  oasie  functions 

1 program 

4/22 

•*ceot 

(excect  FIND  function) 

error 

2200 

0.5 

c o«pe- 

2200 

24 

r i son 

o f *u  1 • 

t > c 1 e 

values 

o r 

range 

4/22 

values 

2230 

-toe  at  steo  1 

1 oroaran 

4/22 

error 

2230 

3.0 

4/23 

1230 

25 

reoeat  steo  1 

1 program 

4/23 

error 

1230 

1 .0 

1 300 

2o 

4/23 

1330 

'eoeat  steo  1 

1 program 

4/23 

error 

1 330 

1.0 

1 400 

27 

4/23 

1430 

recast  steo  1 

2 program 

4/23 

errors 

1430 

1 .0 

1500 

23 

1500 

29 

4/23 

1530 

reoeat  steo  1 

O.K  . 

4/84 

1700 

1 .0 

4/25 

1 100 

1)  Record  when 

errgp 

occurs 

D 1 2 
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ANNEX  D Page  5 of  10 

WORKSHEtT  FOR  DEBUGGING  PHASE 
PROJECT  3 : 4 DEBUG  Run  3 : 5 

Begin  of  Debug  Run  (day/time)  : 04/25/1630 

End  of  Debug  Pun  (day/time)  : 04/25/2400 

3 of  Debug  Steos  incl.  in  Debug  Run:  5 CPU  time  for  Debug  run  (sic):  8.95 
CPU  time  for  necessary  comoiles  (sec)  : 51.3 
a)  11.02  b)  11.48  c)  11.90  c)  9.37  e)  7.53  f)  g) 


Han  nours  for  tnis  Debug  Run  : 5.5  ( i nc ' udi no  oreoarat'C"  o‘  debug  run) 

h 4 N 1 ) 


3T£P 

3 

PROGRAM 

“ART 

OBJECTIVE  AND  EXPECTED 

RESULT 

ACTUAL 

result 

day ; HOURS  I ERROR 
t:«ei'step:  a 

rnMMCllTS 
AN0  COOtO 
ERROR  TYRES 

l ) Pec  c rq  «n?n 

1 

Pino 

Chec  < rlNO  function  tor 

3 oroaram 

4/25 

error 

fund  - 

trivial  cases 

errors 

1 o 30 

1 .0 

OCCurs 

t i on 

Cneck  inout  of  conditions 

1 o 3 0 

30 

4 1 

( s i na 1 • 

(0-4  conaitions  should  oe 

1 700 

31 

C2  7 

di  s- 

stacnec  according  to 

1730 

32 

012 

Crete 

design) 

4/25 

values) 

1730 

2 

^eceat  steo  l 

l oroqran 

4/25 

(using  trace) 

error 

1700 

2.0 

2030 

33 

5 4 (Error  432 

4/25 

not  corrected 

2030 

crocerl y ) 

3 

reoeaf  steo  1 

1 oroaram 

4/25 

error 

2C  30 

0.5 

2100 

34 

C 2 5 

4/25 

2100 

a 

reoeat  steo  1 

I oroqram 

4/25 

error 

2200 

0.5 

2230 

37 

4/25 

2245 

5 

repeat  steo  1 

O.K  . 

4/25 

2245 

1.5 

•i 

4/25 

2400 
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WORKSHEET  FOR  DEBUGGING  PHASE 
PROJECT  S : 4 DEBUG  Run  * : 6 

Begin  of  Deoug  Run  (gay/time)  t 04/25/2400 
Eng  of  Deoug  Run  (gay/tine)  ! 04/26/1430 

a of  Debug  Steps  incl.  in  Deoug  Run;  7 CPU  tine  for  Debug  run  (sec):  10.23 
CPU  tine  for  necessary  conoiles  (sec)  : 67.94 
a)  6.57  b)  12.39  c)  15. bo  g)  o.oo  e)  9.9g  * ) 10.27  g)  9.34 

Nan  Hours  for  this  Oebuq  °un  : 9.0  (including  oreoaration  of  Oecua  run) 

wan  1 ) 


3TE° 

it 

PROGRAW 

“ART 

OBJECTIVE  ANO  EXPECTED 

RESULT 

ACTUAL 

RESULT 

DAY  J HOUPS 1 £P°0P 
TI^EJ/STESJ  3 
1 

CO.wwE’iTS 
AND  C0CED 
ERROR  TYPES 

1 ) Recorg  «n 

i 

A 1 1 

Get  error  f^ee  co^oi  1 • 

3 c o "'O  y 1 e 

4/25 

error 

and  test  ^INO  function 

errors 

2400 

5.0 

occurs 

for  trivial  cases 

4/1  0 

(all  nencers  listeo  oust 

0«00 

39 

A 1 

natch  conditions  soeci- 

0“i  0 

3® 

A 1 

f i si  by  1 nou  t ) 

0“  1 5 

40 

A 1 

4/26 

1000 

2 

repeat  steo  1 

2 orogram 

4/2o 

errors 

1000 

2.0 

1050 

4 1 

C29 

1 100 

42 

C29 

4/26 

1500 

3 

repeat  steo  1 

1 orooran 

4/26 

1.0 

error 

1330 

43 

A 1 

4/26 

1 4U0 

4 

repeat  steo  1 

1 ecnoi 1 e 

4/26 

(including  deouggino 

er -or 

1 630 

1 .0 

aids) 

4/26 

1730 

44 

»1 

5 

repeat  steo  4 

1 orogran 

4/26 

error 

1730 

0.5 

1 745 

45 

A3 

4/26 

1900 

6 

repeat  steo  4 

1 orooran 

4/26 

error 

1900 

0.5 

1 0 1 5 

40 

A 1 

4/26 

1 930 

7 

repeat  steo  4 

O.K. 

4/26 

1930 

1 . 0 

- 
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yiQRKShEET  FOR  DEBUGGING  PHASE 
PROJECT  a : 4 0E3UG  Run  a : 7 

Begin  of  Deoug  Run  (oay/time)  I 04/26/2000 
EnO  of  Debug  Run  (cay/time)  ! 04/26/2300 

a of  Debug  Steos  incl.  in  Deoug  Run:  2 CPU  time  for  Debug  run  (seel:  7.0 
CPU  time  for  necessary  comoiles  (sec)  : 18.14 

a)  R . 7 3 b)  8.41  c)  a)  e)  f>  g) 


Ran  hours  for  this  Debua  Run  : 3.0  (including  oreoaration  g f oecug  "un) 

pan  1 ) 


STEP!  PROGRAM  OBJECTIVE  AND  EXPECTED 

a : PART  RESULT 

• « 

1 • 

ACTUAL 

RESULT 

CAY  1 nOURS 1 ERPCR 
T I y£ 1 / S TE° 1 * 

1 

COMMENTS 
A NO  CODED 
ERROR  TYPES 

• 1 

1 

1 

1 ) Recoro  .ten 

1 • 

• 1 

1 

• 

error 

• 1 

• 

1 

occurs 

1 1-11  1 Test  FIND  function  for 

1 orogram 

4/26  1 

} ! various  inout  (inclucirq 

error 

20001  2.0 

! ( also  enfreme  cong rtigns) 

2 1 2 0 1 

47 

C 2 8 

1 1 (all  oueries  snoull  be 

1 

1 

| ! ans«er*a  acceraim 

4/261 

1 1 oesign) 

2200  1 

1 

• • 

2 ! reoeat  steo  1 

O.K. 

4/261 

• 1 

22001  1.0 

t • 

4/26  1 

1 « 

1 1 

• 1 

• • 

2300  1 

t 

1 

ANNEX  D page  9 of  1° 

HOHKSHEET  FOP  DEBUGGING  PHASE 

f 

PROJECT  a : a DEBUG  Pun  a : 3 

Begin  af  Decug  Pun  (gay/tine)  5 04/27/1500 

End  of  Debug  Run  (day/tine)  5 04/27/IR00 

» of  Debug  Steos  i nc 1 . in  Debug  Run:  1 CPU  time  for  Deoug  run  (sec):  11. RB 

f 

CPU  time  for  necessary  comoiles  (sec)  1 10.02 
a)  10.02  o)  c)  o)  e)  f)  g) 

*an  Hours  for  this  Oebuo  Run  : 5.0  (including  oreoaraticn  of  debuo  run  1 

'••AN  1 ) 


STEP : PROGRAM ; 
a ' PART  1 

• i 

OBJECTIVE  AND  EXPECTED 

RESULT 

actual 

1 RESULT 

OAT ; H0UPS 
TIME ! /STE° 

EPPOP 

8 

COMMENTS 
AND  CODED 
ERROR  TYPES 

i ; an 

• 

i 

Seoeat  Oebuc  Pun  s7  u^oer 
CP/CMS 

( sa^e  results  e*oecte"J) 

O.K. 

a/27 
1500 
4/27 
1 R 0 0 

5.0 

1 ) PecorT  *.nen 
error 
occurs 

annex  0 


Page  9 of  10 


WORKSHEET  FOR  DEBUGGING  phase 
PROJECT  a : 4 DEBUG  Run  * : 9 

Begin  of  Deoug  Run  (oay/time)  : 04/29/1400 

End  of  Debug  Run  (oav/time)  : 04/24/1600 

a of  Deoug  Steos  incl.  in  Oeoua  Run:  1 CPU  time  for  Debug  nun  (seel:  1R.59 
CPU  time  for  necessary  coroiles  (sec)  : R.uo 
a)R.4o  b)  c)  a)  e)  f ) g) 


Han  nouns  for 


STE° 

a 


program i 
part  : 


this  Deoug  °un  : 2.0  (including  orwoaration  of  genua  'uni 

WAN  I ) 

05JECTIVE  AND  EXPECTED  ! ACTUAL  1 C A x \ HOURS  { ERROR  1 CD^'ENTS 

result  : result  iti«E!/step:  a : and  coded 

! ! : : ERROR  TfPES 


I 


All  | various  tests  unaer  I O.K. 

! CP/C HS : ! 

1-Test  initialization  ! 

1 -Test  IISTA  (list  of  all  1 
lattnibutes)  1 

.‘-Test  LI5TC  (1  ist  all  1 

1 availaole  C3«i»an<js)  i 

1-Test  LIST*  (check  for  1 
1 all  menders  ana  illegal  1 
1 i rout ) 1 

1-ChecK  pro  gran  for  inout  1 
1 of  undefined  commands  1 
1-Test  CONTROL  (check  'or  1 
1 ill ega 1 incut  also)  1 

1-Test  functions  Fa  ana  1 
1 ATTR  for  several  me-ters  1 
1 in  control  1 

1-Test  SiilTCm  (en«ihe  out-1 
1 out  f i 1 e ) 1 

!-Tes t K£y  (change  of  <ev  1 
! attrioute  as  desired  by  1 
1 u s e r ) ! 

1-Test  LISTCBASE  (al I »e«-  ! 

! bers  are  listed  with  all  ! 

! all  attribute  value  i 

1 pairs)  1 

1-Test  LI3IALL  (memoers  are! 

1 listed  only  by  <ev  attri-1 
! cute)  1 

1-Test  F I NO  function  (using! 

! various  inout  comci-  1 

1 n a t i o n s ) 1 


4/26  1 
14QC1  2.J 


4/2? 

1600 


I ) 


"ecord  wnen 

error 

occurs 
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*orksheet  for  debugging  phase 

PROJECT  a : 0 OEBUG  Run  a : 10 

Begin  of  Qeoug  Run  (oay/time)  : Ou/26/loOO 

Eng  of  Debug  Run  (cay/time)  : 04/23/1800 

a of  Debug  Steps  i no  1 . in  Oeoug  Run:  1 CPU  time  for  Debug  run  (sec):  12.09 
C°'J  tine  for  necessary  compiles  (sec)  : °.4b 
a)  R.oo  o)  c)  c)  e)  f)  o) 

fan  hours  for  this  Debug  Run  : 2.0  (including  preparation  of  cecua  run) 


•AN  1 ) 


stepjprogram 

a • PART 

OBJECTIVE  AMO  EXPECTED 

RESULT 

ACTUAL 

RESULT 

OAT  :~0UPS!ERR0R 
TIfE!/ST£9!  a 

COMMENTS 
A NO  COOED 
ERROR  TYPES 

• 

• 

1 ) Pecoro  *hen 

• 

error 

• 

1 

• 

OCC'-/rs 

i : a 1 1 

Various  tests  unoer 

4/291 

• 

• 

CP/C.vS: 

laoo:  2.o 

i 

a)  Test  SU  (super  user 

1 oroarair 

» 

■ 

• 

i 

f unc  t i on ) 

error 

1620! 

49 

C2? 

• 

i 

t> ) Test  FIND  function 

• 

• 

for  0 conditions  (all 

O.K. 

• 

1 

• 

« 

memoers  should  ce  listed) 

1 

1 

• 

i 

i 

c)  test  FIND  FIND  ID 

• 

1 

1 

• 

• 

using  1-4  different  con- 

O.K  . 

1 

1 

i 

t 

ditions  (check  correct- 

1 

1 

• 

• 

ness  o*  orognan  especially 

i 

1 

i 

multiple  values  and  range 

i 

• 

i 

i 

• 

values) 

1 

i 

• 

i • 

d)  Test  *£Y  function  for 

» 

• 

1 

• 

i 

illegal  mouts  (proaran 

O.K. 

• 

l 

■ 

• 

should  provide  an  acoro- 

1 

• 

• 

t 

oriate  error  message) 

1 

) 

i 

• 

9X2?  1 

% 

• 

i 

i 

i e so ; 

i 

• 

2to6 
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Page  1 of  2 


PROJECT  # : a 


ERROP  LISTING 


Begin  of  Project  (dav/time) 
End  of  Project  (day/tine) 
Man  hours  for  total  project 


04/ 1 P/ 1 400 
04/03/1800 
101.0 


E*«08 

4 

PHASE 
in  which 
ERROR  was 
Hi  S- 
covered 

phase 

in  m n i c *> 
SPRQw  was 
naae 

ERROR 

Tr=E 

f see  ANNEX  F) 

TP-E 

scent  to 
solve  the 
ERROR 
(Man 
h i n . ) 

i 

Debuddi ng 

Coding 

Cb 

S 

2 

Deouggi ng 

Coding 

A 1 

5 

3 

Debuggi ng 

Cod i nq 

A 1 

5 

4 

Oebugg i ng 

Codi ng 

A 1 

5 

5 

Oebugqi ng 

Coding 

A 1 

5 

6 

Codi nq 

Cod i na 

CP 

5 

7 

Debuggi ng 

Cod  1 ng 

C 1 7 

5 

3 

Oeouag i nq 

C O d 1 n g 

C 1 2 

5 

9 

Debuggi ng 

Coding 

C7 

s 

10 

Debuggi ng 

Codi ng 

A 1 

s 

1 1 

Decugai ng 

Design 

D 7 

s 

12 

Debuggi nq 

Coding 

C20 

5 

13 

Oebuggi ng 

Coding 

C 2 1 

15 

1 4 

D*buqq i ng 

Coding 

C21 

15 

IS 

Debuggi na 

Cod i nq 

C20 

10 

lb 

Debuqg i nq 

Oes i gn 

012 

25 

17 

Oebugg i ng 

Coding 

Cb 

5 

1 8 

Debuggi ng 

Coding 

C 1 7 

5 

1° 

Debuoa i ng 

Design 

D 1 2 

30 

20 

Debuqg i nq 

Codi ng 

C2  7 

10 

21 

Oebuggi ng 

Coding 

A 1 

5 

22 

Oebuggi ng 

Coa 1 ng 

A2 

5 

23 

Deouaa i ng 

Codi ng 

C 2 3 

5 

24 

Oebugg i ng 

Design 

D 1 2 

10 

25 

Oebugq  i ng 

Design 

03 

15 

2b 

Oebugg i na 

Design 

0 1 3 

30 

27 

Oebuggi nq 

Design 

D 1 5 

20 

20 

Deouggi nq 

Design 

D 1 2 

30 

2R 

Oebugg i nq 

Coding 

C27 

5 

30 

Oebugg i ng 

Codi ng 

A 1 

5 

- 2b  7 


a of  CTh£R 
STATEMENTS  Ok 
PARTS  QP  ThE 
PROGRAM 
AFFECTED 


Page 


TIME 

soent  to 
sol v*  t n e 
ERROR 
(Man 
min.) 


31 

Deouaa i ng 

Cooing 

C27 

10 

32 

Oeougg i ng 

Design 

012 

40 

33 

Oeouog i no 

Deouga  i na 

34 

15 

34 

Oeougco  i ng 

Coa i ng 

C29 

15 

35 

Coa 1 nq 

Coai na 

A 1 

5 

36 

CoOing 

Cooing 

A 1 

5 

37 

Deouqg i ng 

Ceougai na 

A 1 

5 

33 

CeOuqg i ng 

Cooing 

*•  l 

5 

39 

DeCugqi no 

CoOing 

A 1 

5 

uO 

Oeouqc i ng 

CoOing 

A 1 

5 

41 

Oecugo i nq 

CoOi ng 

C2i 

5 

42 

Oebugg i ng 

Cooing 

C 2 3 

5 

4 3 

Oeouqg i nq 

CoOing 

11 

5 

44 

Oeougg i ng 

C 00 i ng 

A l 

5 

45 

Oeouqo i na 

Oeougg \ ng 

A3 

5 

46 

Oeougg i nq 

Codi ng 

A 1 

10 

47 

Oebuqg i na 

Codi ng 

C2e 

10 

43 

Cod' nq 

Codi ng 

C 17 

5 

40 

Denugai ng 

CoOing 

C 2 7 

30 

50 

Testing 

Design 

03 

120 

ANNEX  El 

ERROR  LISTING 

PROJECT  # : a 

Begin  of  Project  (dayXtime)  : 04/19/1400 
Eng  of  Project  (oay/time)  : 04/05/1300 
*an  hours  for  total  Drojeet  : 101.0 


E^ROR 

PHASE 

PHASE 

9 

in  which 

in  »h  i e h 

ERROR  .as 

ERROR  .as 

di  S- 

made 

covered 

ERPCR 

TYPE 

t see  ANNEX  r ) 


of  2 


s of  OTHER 
STATEMENTS  OR 
PARTS  OF  ThE 
PROGRAM 
AFFECTEO 


ANNEX  E2  Page  1 of  2 

ERROR  LISTING  (COMMENTS) 


ERROR 

tf 

DAY 

TIME 

COMMENTS 

(Evidence,  thoughts,  «hy  has  the  error  made? 
WHY  AND  HOfl  HAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 

i 

04/21 

1030 

Errors  *1  triru  5 are  cue  to  influence  of  fatigue. 

2 

1030 

3 

1030 

a 

1030 

5 

1030 

b 

1830 

Founrs  wn  ile  c*ec*ing  orevicuslv  written  co'le  auring 

7 

1O00 

ounc'ing  of  cares, 

0ifferer>t  variaole  assumed. 

3 

2000 

Misunq  nanoatory  oee'aration. 

<? 

2130 

10 

2130 

1 1 

2200 

12 

04/22 

1000 

1 3 

1020 

14 

1030 

Errors  *14  t n ru  lo  are  oue  to  insufficient  oes  < creeping. 

15 

1 100 

16 

1110 

17 

1200 

13 

1230 

IP 

1430 

20 

1500 

21 

1 °45 

22 

1050 

Uooer  case  <ev  or»ssea  w^ile  ounening  caras. 

23 

1055 

Usage  of  "ThEN“  insteaa  of  "DO". (wrong  fomat) 

24 

2200 

25 

04/23 

1230 

26 

1300 

27 

1400 

23 

1500 

20 

1500 

30 

Ou/25 

1630 

II.  .1  •M.IWi  ■ " ■" 


I 


ANNEX  E2  Page  2 of  2 

ERROR  LISTING  (COMMENTS) 


ERROR 

0 

DAY 

TIRE 

COMMENTS 

(EVIDENCE,  THOUGHTS,  WHY  w AS  THE  ERROR  MAO E? 
WHY  ANO  HOW  WAS  THE  ERROR  DISCOVERED? 

ERROR  BLOCKING,  etc.) 

51 

04/25 

1700 

32 

1730 

No  orooer  desk  checking.  (Error  could  have  ceen 

33 

2030 

avoided  by  des«  checking.) 

Inaooroori at e correction  of  error  s32.  Debugging 

3a 

2100 

results  »ere  not  interoreted  correctly, 
forgot  ten  * 1 1 ’ . 

35 

2100 

Errors  035  ana  36  are  due  to  f a t i que • ( t r i v i a 1 errors] 

36 

2200 

37 

2230 

3? 

04/2o 

0900 

Errors  »38  thru  40  are  due  to  lack  of  c one en t r a t i on 

3R 

0910 

while  ounchinq  cards. 

40 

0915 

a t 

1050 

u2 

1100 

43 

1 330 

44 

1730 

as 

17a? 

4 6 

1815 

47 

2120 

OouD  1 e negation  in  connection  with  logical 

as 

04/27 

2050 

could  have  oeen  tes tec  Cv  usage  of  a trutn  taDle. 
The  error  could  Have  oeen  avoided  this  wav  during 
a desk  test. 

Found  while  Ounching  cards. 

aR 

04/28 

1620 

50 

05/01 

1700 

It  was  suronsinq  to  the  e*oeriment  orogrammer 

that  using  the  E3CDIC  character  set  the 
following  conoarison  will  oe  evaluated  "TRUE": 

Error  9 50  was  ^ade  cecause  this  relatiorsnio 
was  not  known. 

270 


1 I «.  mW.MTf<hn 


J 


annex  f 


final  STATISTICS 


PROJECT  a 4 


final  statistics 

Project  name  : DATA  RETRIEVAL  STSTEm 
Short  description: 

The  D rogr am  is  aes i gned  tor  us  aoe  urdtr  C P / CUS . It 
eioects  an  inout  tile  labeled  "DBASE  Input"  to  contain  cats 
base  information  in  a oarticular  format  suer  trat  t"e  oro” 
gran  can  rjao  all  information  and  store  it  into  'emery. 
After  a I I bate  case  mergers  are  ~.e4  i neb  ov  incut  ana  ’ tio!  a* 
rented  m thin  linked  lists  the  user  nav  soe'ate  ucon  tie 
bata  oase  using  functions  from  a previously  be  f i neb  set  of 
f unc t i ons  . 

The  o rogr am  has  been  oesigneb  arc  i rol emented  in  sue  i a 
manner  that  it  » i 1 1 oe  easy  to  irole«*nt  more  functions  or 
to  extent  the  Currently  helmed  limits  of  the  cregram. 

Quantitative  measures: 


1 . 

a of  source 

st  at  e«ent s 

: 1 084 

2. 

Total  man  hours  for  oroject  : 

101  . 0 

3. 

^an  hours  scant  in 

a ) Oes i gn  : 

2a. 0 

b)  Cooing  : 

24.5 

c)  Deougging 

: 4 1.5 

b ) Testing  : 

11.0 

a . 

CPU  time  for 

comoi l es : 

343.57 

sec  • 

5. 

CPU  time  for 

debug  runs 

: 14  0. 

05  sec  • 

o . 

CPU  time  for 

test  runs: 

38.04 

sec  . 

7.  a of  test  and  debug  runs:  13 

8.  a of  test  and  debug  steps:  43 
R.  a of  errors  found:  50 

10,  Total  man  hours  used  to  correct  errors:  11.25 


ANNEX  F 


FINAL  statistics 


11.  Error  Detection: 

Mean  time  between  error  detections:  87.7  man  min. 


* of 
errors 
T 

„ i 


JO 


25 


20 


15 


10 


5 ! * » •*» 


j * * * * * * * * 

l ; * * * * *****  * * ***  ***  ***  man  hours 

1 2 3 a 5 6 7 3 9 10  1 1 12  13  oroject  t 

time  between  ekror  DETECTIONS 

(measured  fro*  ooint  in  time  of  detection  of  orevious  error) 


*e 


annex  f 


FINAL  STATISTICS 


12.  Error  Correction: 

Mean  time  to  correct  an  error:  13.3  man  min. 

* of 
Errors 
correct  ed  . 
t 

I 

; * 

30*  * 


25 


20 


15 


- 273 
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FINAL  STATISTICS 


13. 

When 

errors 

Mere 

f ound  5 

a) 

a 

of 

errors 

f ound 

during 

design  chase: 

0 

— 

0.0 

b) 

a 

of 

errors 

found 

during 

design  rev i t» : 

0 

0.0 

c) 

a 

o f 

errors 

found 

during 

codi nq: 

4 

z 

8.0 

a) 

a 

o f 

errors 

f o<jnd 

du  r i ng 

debuggi ng : 

45 

= 

90.0 

• ) 

a 

o f 

errors 

f ound 

during 

w r i t i h g of 

test  procedures: 

0 

0.0 

50 

f ) 

a 

o f 

errors 

f ound 

du  r i ng 

testing: 

1 

= 

2.3 

la. 

«hen 

errors 

mere 

nade: 

a ) 

3 

o f 

errors 

Tiade 

curing 

lesion  erase: 

10 

= 

20.0 

o ) 

3 

o f 

errors 

■naae 

du  r i ng 

de  s i on  review: 

0 

= 

0.0 

c) 

3 

o f 

errors 

•race 

Curing 

coding: 

37 

z 

7a. 0 

0 ) 

3 

o f 

errors 

naoe 

du  r i ng 

deoucg  mo: 

3 

= 

o . 0 

e ) 

3 

o f 

e r rors 

naoe 

du  r i no 

writing  of 

test  orocedures: 

0 

= 

0.0 

f ) 

3 

o f 

errors 

naoe 

curing 

testing: 

0 

z 

0.0 

50 
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FINAL  STATISTICS 


15.  T I ME  HISTORY  GRAPHS  : 


a of 
Errors 
T 


1 


a 


A 

A 

A 

A 

C 


ao 


Error  T yoes : 

A:  Clerical  e r r o r 
9:  Deougaing  e'ror 

C : Cooing  error 
0:  Design  error 


C 

A 

DCC 

c 

0 

A A A 

CACCC 

A 

oc 

C C AC  A 

CCCCOD 

AO 

DO 

A06AAC AA 

uO  sO  90 


CC  0 


100 


nan 

nours 


MUMPER  OF  ERRORS  F0UN0  VS  “ROJECT  time 


a of 
Errors 


10» 


* * * * 
* * * * 


a ao 


aO 


bO 


60  100 
number  of  errors  cor°ec teo  vs  project  TIME 


► Yian 

■'Ours 


27S 


ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


1 . 


Design  Errors 


The  foilwing  tyres  of  errors  acoly  to  Doth  categories 
"System  Design  Errors"  and  “Program  Design  Errors": 

Dl  : Communication  Error 

02  : Design  Negligence 

03  : forgotten  Cases  or  Steos 

Du  : Timing  Proolems 

OS  : Errors  in  I/O  Conceots 

Do  : Data  Design  Er-or 

07  : Ini t i a 1 ' /at i on  Error 

D8  : Inaaeouate  Checking 

DR  : Extreme  Conditions  Neglected 

010s  Seouencinq  Error 

Dll:  Index i ng  Error 

012:  Loco  Control  Errors 

013:  Pisuse  of  boolean  Expression 

Diu:  Patnematical  Error 

D15:  Peo resen t a t i on  Error 

Dio:  M i s unoe r s t ang i ng  of  Proole”  Soec ' f ' c at ' ons 

017:  Other  Design  Errors 


2 . Coding  Errors 


Cl  : M i sunge r s t ang i ng  of  Design 

C2  : Negligence 

C3  : I/O  Format  Error 

Cu  : “isolace:  Oata  Declaration 

C5  : Pvjlttole  Data  Declarations 

Co  : yissing  Oata  Declaration 

C7  : Inadeguate  Data 

C8  : Ini t i a 1 i tat i on  Error 

CP  : Error  in  Parameter  Passing 

CIO:  Inadeguate  or  Forgotten  Checking 

C 1 1 : Level  Prob 1 ers 

C 1 2 : Pissing  Declarations  of  block  Limits 

C 1 3 : Case  selection  error 

Clu;  GO  TO  Problems 

CIS:  Comment  Error 

C 1 to : Forgotten  Delimiter 

C 1 7 ; Inconsistency  in  Namina 

CIS:  flrong  Use  of  Nested  IF  Statements 

C19:  Indexing  Error 

C20:  Inconsistent  Use  of  Variables  or  Cata 

C2I:  Seauenc l ng  Error 

C22:  Flaa  Usage  Problems 

C23:  Syntax  Error 

C2u:  Looo  Control  Error 

C25 : Incorrect  Exit  from  Subroutines 

C2f>:  Language  usace  Problems 


27b 


ANNEX  F 


ERROR  CATEGORIES  AND  TYPES 


C27:  Forgotten  Statements 

C2S:  Reoresent at i on  Error 

C2qs  Control  Sequence  Error 
C30:  Incorrect  Subroutine  Usage 

C 3 1 : Other  Coding  Errors 


3.  Clerical  Errors 

A l : Manual  Error 

A 2 : Mental  Error 

AS  : Procedural  Errors 

A 4 : Other  Clerical  Errors 


Debuaging  Errors 


91  : I naooroc r i a t e Use  of  Debugging  Tools 

92  : Insufficient  or  I n aco r oc r i a r e Selection 

of  Test  Cases  or  Test  Data 

93  : m j s j n t eror er a t ' on  of  Deouoging  Results 

94  : ui si nteroretet ion  of  Error  Source 

95  : Negl i gence 

?o  : Other  Deouaoing  Errors 


5.  Testing  Errors 


T 1 ; Inaaeauate  Test  Case(s)  or  Test  Data 

T2  : smteroretat  i on  of  Test  Results 

T3  : M, j j n t e rcr e t a t i on  of  Proaram  Scec i f ' c at i on 

T4  : Negligence 

T5  : Dtner  Testing  Errors 


?77 


ANNt<  G 


®ace  1 o*  Sh 


PROJECT  = : 
Program  Dart 


DIRECTED  GRAPH  REPRESENTATION 

CLEAR  IN,  CLEAROUT,  CARDIN,  PRINTOUT, 

TYPIN,  TyPOUT,  RILE  I/O,  TERMINAL  I/O, 

GET  OLO  FILE  INFORMATION,  .y-ITE  NE.y  FILE 
INFORMATION,  CAR),  CAR2,  COB  1 , CD3?, 
SETCAR1,  SE  T C A R 2 , SETCD0!,  SETC0R2,  FREE  5 , 


FRF£2,  find  Ir 


5 E T 0 y A L . E , •' a I ' 


COmPlEyITy  mcasuREr: 

NUMBER  OF  STATEMENTS:  2 
NUt-eSR  CF  NCCES:  2 

NUMBER  CF  ARCS  : 1 

(NUMBER  CF  PATHS:  1 

CYCLCYATIC  iNUNBER:  V ( G 


REACHABILITY  CF  RCCES: 
RCCE  l : 1 

NCCc  2 : I 

SLY:  2.CC0CC0 

reachability  ince* 

CF  OIRSCTEC  GRAPH: 


1 .CCCCQC 


NOTE:  A))  23  sutrsutires 


sjr*  StrjCNji 
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ANNEX  G 


ace 


3l«ECTE0  r^-Pu  JEpot5E''*T^TIj. 

3 P 0 J E C T 3 : *J 

pfo qrj-n  cart  : 5pACE 


COMPLEXITY  ^EASjFES: 
NUMbE®  UF  S r A TEME  « r S : -* 

\LV9EF  CF  NC  C £ 3 : * 

\u^9=R  CF  ARCS  : 4 

,\  u s g 5 R CF  R A T r S J 2 

CYCLOJT  IC  NoMEe=  : v ( : ) = 


ScJCl-iaiLlTY  C F NlCeS: 

sees  i ••  ! 

\c:e  i ••  4 


o.CCJJwO 

; = :Ci-4  6 !L  tTY  ! N C : 

;f  : iscCTic  a?***- : 


, ;c:jc: 


0 


A V»C  < 


a : 

i,T*  r»r’ 


: 1 . 1 * He  1 2C  -LL 


C3I*BLE  *It. 

-eiSu-iES: 

•*l  M*€® 

S r a f c >4  f i r s : 

'.f 

• - - : S : * 

: c 

i;;2  : ii 

■ » - - . - 

: ;c 

vu*ie<  : 

MV 

'w.i 

StJ 

i!!I 

Yn\ 


Jiii.:*' 

i i 

j 1 

a | 
« I 
1 I 

a i 

\ i 


. » : 


;*  v::£i: 

J 

4 


:•.;! « 

■ i - i. 


4 4 ...  4 L 


► 3t  . T J ' ,T 


ANNE  < IJ 


<3  B A p h b£wPES£jTA1 


ppnjECT  = : 
?roQ',a™  cart 


COMPLEXITY  v£ASU»E3: 

NUM9E«  OF  STATEMENTS:  E'J 
NUFSEB  Cc  NCCES:  6 

Mj  f*  3 E 3 CF  ABCS  : 3 

\UF3c3  CF  FATES:  9 

CYClCJOTIC  NUECES:  vU)* 

BEACrAE lilTY  CF  NCCES: 
NCCE  1 : i 

NCCE  l ■ 1 

NCCE  3 : j 

\c:s  t : i 

NCCE  E : 1 

NCCE  4 : 1 


SOI':  e.CCCCCC 

aeacea5;l:*y  incex 

CF  C '.3  EC  *SC  SSAPF : 


i.crcoco 


-aa»  5 c f 5- 
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ANNEX  G 


Paae  7 of  5 ^ 


DIRECTED  GRAPH  REPRESENT  A I ION 


PROJECT  o : a 

P roa ram  part  : DIAGNOSE 


complexity  measures: 

number  OF  statements:  S3 
AUMB  Efi  CF  ACCES:  16 

NUMBER  CF  ARCS  : 23 

AUABER  CF  PATHS:  AO 

CYCLCMATIC  NUMBER:  V ( G > = 

REACHABILITY  CF  ACCES': 


SUM:  IS  1 .3030 

REACHABILITY  I.NCEX 
CF  CIRECTEC  GRAPH  : 9.Ra75C3 
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' a g ? 9 O'  A - 


O.’PECTEO  g 4 a ? h iP  = n£  sE-i  r a T ION 


PROJECT  =>  : 4 

Program  o.jrt  : E<TPaCT»  PILc>  LIT 
FI  NO  VALUE  LE’;G7> 


LI5TC.  LIST -'LL, 


COMPLEX  TTY  '■'EASuhES: 

MUMPER  OF  STA rEvEMTSs  9-13 

MJF9ER  CF  NCCcS:  5 

NUF 8 ER  CF  ARCS  : 5 

NUFSER  CF  PATHS:  2 

CYCICYATIC  N U M 8 S A : V ( £ ) = 


PEACPASILITY  CF  NCCEE: 
NCCE  l : 1 

NCCE  2 : 2 

N C C c a : * 

NCCE  J : 2 

NCCE  ; : 2 


Eli,*:  3.CCCCOO 

PE  AC  r A 8 iC  TTY  INC5X 
CF  CISECTEC  £RAPh  : l,:S«99S 


?85 


ANNEX  0 


Pace  II  o * iJ 


directed  graph  represent  a t:on 

PROJECT  = : -i 

Ppoaea'fl  cart  : STORE 


COMPLEXITY  ■-'ENSURES: 
NUMBER  OF  STATEMENTS:  1° 
NURSES  CF  i\CC  ES  : 7 

NURSES  Zr  ARCS  : 3 

NUNSER  CF  PATHS:  7 

CYCLCPAT  IC  NoMEES:  V(G)  = 


REACHABILITY 
NCCE  1 

NCCt  2 

NCOS  2 

NCCE  A 

NCCE  5 

NCCE  6 

NCCE  7 


CF  NCC 
7 
e 


6 

6 

7 


SUM:  33.  CC 

REACHABILITY  INC?  X 
CF  C IRECTEC  GRAPH  : 


JOO 


5 . * 2 65  7 1 


= aae  12  of  5- 


OIRECTtD  GRIP*  PEPPf  S£  T 10'; 

PROJECT  s : u 

?r55ra»  oart  : GET  ATT-JtB.irES  A'iO  VALUES 


a)  s of  nodes:  ° 

b ) 3 of  arcs  : 1 ! 

c)  3 of  staton«"ts:  i 
o)  3 of  cat-s:  * 

e ) Reac  r ac  t H t / : * 

f)  Cvelooatic  otjrr»r:  a 

* Nun’cer  of  oaf's 

ana  ffscnao; I if» 
are  very  Jar c e . 
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A N M t X G 


OIOECTED  GP upw  PPPOF3E-.TAT 

PROJECT  a : 4 

Ppoaram  cart  : GET  ATTRIBUTE 


COMPLEXITY  '-'EASUPES: 

NUMBER  OF  STATE WE' ITS:  23 

AUi'S  cP  CF  NCCES:  13 

NUP3  ES  CF  ARCS  : 13 

SU  PS  EP  CF  FATFS:  16 

CYCLCMATIC  NUM8E3:  V(G)=  5 


Rc AC  FACILITY  CF 


NCCE 
NCCS 
SCCE 
N CCS 
■SCCE 
SCCE 
SCCE 
SCCE 
SCCE 
SCCE 
SCCE 
NGCE 
NCCE 
SC  CE 
SCCE 


1 


1C 
11 
12 
12 
L A 
15 


SCCE3: 

1 
1 
1 
1 

2 
2 
A 
A 
A 
2 
6 
6 
t 

12 

12 


SUP:  a A . CO JOO 

P SAC  FA  6 I u I TY  INCEX 
CF  CIFECTEC  CPAPf:  A#26c666 


annex  r. 


Face  t S o 


OrKfCTE'l  r, SSPH  “EPEESE'iYAT  ION 

PROJECT  e : 4 

Program  oart  ! F" I ’4 *">  ATTRIBUTE 


COMPLEXITY  MgASuWES: 


NUMBER  OF  3T  A T£  V£'|  T 3 : l i 


NUMBER  C r N C C c S : 
NUMBER  Cr  ARCS  : 
NUMBER  Cr  FATES: 
CYCLCMATIC  NUMBER  : 


£ 

■ 7 
A 

V t G ) = 


REACFABILITy 
NCCE  l : 

SCCS  2 : 


N CCS 
NO  C£ 
NCCE 
NG  C E 


A 

e 

c 


CF  NOCSS: 
1 
2 
1 


A 


SUM  : 12 • CCOCO 

REACrABlL  I T Y INCE.X 
Cr  CIRsCTEC  GPAPr: 


3 


* 54 


2.CCCCOO 


ANNE  < i» 


P aae  1 o n*  3-* 


OIRECTEO  ',PAPW  REPRESENTATION 

PROJECT  a : 4 

Pro  grii.  oaft  : SKIP,  FT'.;)  CCV-'A-iP  , 'SET  CONDITIONS# 
TEST  ALL 


COMPue*  I Tr  -‘EASURES: 


NUMBER  OF 
MJPSER  Cr 


STATE'- 

NCCES 


NOSES  Cr  ARCS  • 
NURSES  CF  °AThS : 
CYCLCMATIC  NURSES t 


2 

v IG  I- 


:z 


SEACFA8  IL 
NGlE  1 
SCCE  2 


NCCE  h 


ITV 


CF  NCCES: 
I 


SUR : 6.CCCC0C 

PEACrA  e li.  ITV  INCSX 
CF  0IS8CT8C  GrAPP  : 


l . see J CO 


ANti£<  G 


Price  I ° O*  3 A 


DIRECTED  OPAPw  ® £ P P E S E Ti  T A T I G * 


PROJECT  = : J 

Program  o a <•  t : LI3T0“43E 


COMPLEX  Tty  .VE4SURE3: 
'1UMSER  CF  STATEMENTS:  11 

\biv9 E?  CF  NCCES:  i 

NLE8ER  CF  ARCS  : 7 

,\UE9£R  CF  FATES:  A 

CYCLCYATIC  NUV3EF:  V(G)= 

R S AC  EA  8 I L I TY  CF  NCCES : 


SLY:  U.CQCCC 

A SACEA  a IL  !TY  ’.NCSA 
CF  DiSSCTEC  GFAFE:  3.664666 


29b 


A N >\  t < G 


DIRECTED  G y a P w cEPy(:5E'iTATiC‘< 

PROJECT  c : 1 

P rogra.n  car  t : HSISWCPT 


G 


c 3Ca  22 


OiQFCTEn  G « A P h 3EPCF!St‘jTATI0:4 

PROJECT  = : <J 

Program  cart  ! Ll$l‘'7TiJtBUTE 


a ) 

a 

o f 

noies:  l o 

o ) 

a 

o f 

arcs  : 2 l 

c ) 

= 

0 f 

s t a »■  e^en  t s : 3-* 

a ) 

a 

o t 

car  ■>  a : * 

e ) 

- 

a ac 

''anility!  * 

f ) 

r 

y c ' 

omatic  m .j  - n a r : 7 

« \gmcer  0 * o a f * s 
ara  reac^ar  i 1 > • v 


PROJECT  * : 

Program  oar( 


COMPLEXITY  '•'EASUkES: 

NUyBE P OF  STATEMENTS:  la 
MIESES  CP  NCCES:  IE 

MISSES  CF  JSCS  : 2A 

NUMBER  CF  PATES:  Lc 

CYCLC.yATIC  MSEEF:  V ( G ) = 


S6J< 

:ea2 IL  I TV 

SCl 

S l 

NCC 

: 2 

\CC 

- - 

NCC 

4 

n;; 

: - 

NCC 

E 6 

ncc 

E 7 

\CC 

e a 

see 

E A 

NCC 

E 1C 

NCC 

E 1 1 

\CC 

E 12 

NCC 

E 12 

NCC 

E 1A 

NCC 

E 15 

NCC 

E It 

NCC 

E 17 

NCC 

£ 1 a 

NCC 

E IS 

SO*': 

ScJCeASILITY 
C F OZPECTcC 


Cr 


NCCES  : 
l 
I 
I 
1 
1 
1 
1 


a 

A 

A 

A 

A 

A 

A 


8 

It 

It 


a- .C30JD 

INCEX 
GRAPE  : 


OIAECTEO 
: COMPiPfi  1 


A ■ 268A2 I 


ANNEX  G 


Pace  Bo 


CIPEC'FO  G SAP*  ?FPRf  SE''1  TA  T ! 0-< 

PROJECT  s : “ 

Proqra'  oa't  : “A 


OMPLExITT  '•'EASURES: 

IU««ES  OF  ST  A TE“ENI5  : 

,UFSER  CF  NCCES:  iO 

UF8ER  CF  A*CS  : il 

iUFBER  CF  cATFS:  3 

'vri  r«ATir  NLNBtx  5 V ( G I s 


; 15. COOCQ 

5 ? iSCK  A 0 IL  ITY  INC  c X 


A N <Ni  £ * G 


PROJECT  a ; 

pfogra*"  c a r r 


nrp£cr£o  r,p 

GET  PEGUE3T 


\P~  JtPPESE 


a ) a of  noies : 1 2 

*3 ) a of  arc  s : > ^ 

c ) 3 of  s r a r e"e«t s : 

o)  * of  oa  r ^ s : * 

»)  Peac  ^ ar  H i r v ! * 

*)  Cvc^oTaf^c  ^■j'-ce1 
* >i  u rr  o e r ? * cat’'1? 

am  reacnapH  i *v 

are  verv  1 a r -j e . 


3 


A.-4NtX  G 


^ace  SO  "i  * S 4 


DIRECTED  r.R43M  =E°WESE'I  r AT  ION 

PROJECT  a : a 

PfoqfaTi  cart  : TEST  CONDITIONS 


CO” PLExITT  JE  4SUPE  S : 

>4(jMO£B  Ojf  STATEMENTS:  19 

MjHSSa  C F NCCES:  1J 

NU“6ER  CF  ASCS  : 12 

NUH3ES  CF  PATHS:  a 

CTCLCHAT IC  NCM3ES:  V<2)  = 


9 £ AC  HA  3 

ILITY  CF  \C2ES: 

\CZz 

1 

i 

N2CE 

2 

2 

NCCE 

2 

\CCE 

A 

2 

NCCE 

3 

2 

NCCE 

6 

2 

NCCE 

7 

2 

nCCE 

c 

NC  LC 

s 

A 

NCCE 

iC 

4 

. SUV 

: 

2R.COOCO 

5SACHAS  It.  ITT  IMCEX 
CF  G ISECT  EC  GRAPH: 
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A N i’l  E X G 


P aqe  31  of  3 a 


DIRECTED  O&APP  REPRESENTATION 

PROJECT  a : a 

Program  oart  : COMPARE  "ANY 


a)  a of  nodes:  12 

b)  a of  arcs  : 1“ 

c)  a Of  statements:  27 

d)  a of  oaths:  * 

e ) Reachability:  * 

f)  Cyclomatic  number:  a 
* Number  of  oaths 
and  reach  ad i 1 ' t v 
are  very  lame. 


30? 


4.'iNE»  r. 


o I°EC  fE  o n-ip-  »fppp36'  r j r il'v 

PROJECT  = : u 

Proqrair  oaf?  : LL'^If 


complexity  measures; 

MUM8ER  OF  S T i TEslEO  r S : 15 

MIFBER  CF  NCCES:  7 

NUF65R  CF  ARCS  : 4 

NUMBER  Cr  FATl-S:  7 

CYCLCV4TIC  NLiMEER  • v«C)=  3 


SL,V : 3E.C0C00 

RE4CF4E !LI7Y_ INCSX 
C F CIREC7EC  GRAFF! 


5.1RZ357 
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Pace  55  o * 5 4 


DIRECTc'1  r.»4pp  REPRESENTATION 

PROJECT  3 : J 

P r c a e a 4 o a r t : J L I '"I  T 


COMPLEX  I T Y ■••£  A SURE  3 : 

NUMBER  op  STATEMENTS:  W 

NURSE?  CP  NCCcS:  1C 

.NUMBER  CF  ARCS  : 1C 

MJF6ES  CF  FAThS:  la 

CYCLE. mat  IC  NuMSEF  : v l G ) = 


REACFAiiUTY  CF  ,\C 

NCCE  1 

I 

vc:s  2 

2 

NCCE  j 

1 

NCCE  4 

2 

NCCE  ; 

14 

NCCE  c 

1C 

NCCE  7 

t 

NCCE  B 

u 

NCCE  9 

14 

NCCE  1C 

14 

x(..m:  7 £ . CCCCC 

REACHABILITY  INCEX 
CF  OIF.  EC  TEC  TRArr: 


7. =99999 


V 


ANNE < G 


OI»£CTEO  graph  REPRESENTATION 

PROJECT  p : u 

Program  oarr  : SWITCH 


COMPLEXITY  measures: 


NU«8EP'  OF  STATEMENTS!  IS 
f^UYSES  CF  KCC'zS:  4 

MJYBER  CF  ARCS  : 4 

NLY3ER  CF  FAIRS:  2 

CYCLCMATIC  NUMBER  : V l G » = 


2 


REACHABILITY  CF  NCOES: 
NuCS  1 : 1 

SCC£  2 : l 

NuCS  i : 1 

\CC£  4 : 2 


SUM:  S.CCCOCO 

R E AC  R A 3 ILITY  INCEX 
CF  CIRECTEC  GrAPr:  L.CECCCu 


aae  o * 5* 
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annex  h 


Page  1 of  3 


TEST  PHASE  DESCRIPTION 

Project  # : R 

Test  run  « : 1 Including  5 Test  Steos 

Begin  of  Test  (day/time)  : 04/2R/1330  End  of  Test  (aay/time)  : Gp/29/1530 

CPU  time  for  necessary  eomoiles  (in  sec.  5:  R.OA 

a)  R.OR  b)  c)  d)  e)  f)  g) 

CPU  time  for  TEST  run  (sec)  : 12. S 

“sn  Hours  for  this  Test  run  : 2.0  ( i -e 1 udi nq  oreoaratior  of 
tests) 


1 ) 


TEST 

step 

OBJECT  vE 

EXPECTED 
°E  SUL  T 
(TOLERANCE) 

actual 

RESULT 

ERROR ! DA  i 
a l T I v E 

1 

1 

COMMENTS 

AND 

CODED  ERROR 

types 

l 

Test  of  I/O  func- 

I/O  can  be  direc- 

0.*. 

1 J/2R 

1 ) Record  -hen 

ions  unaer  C?/C*S 

ted  as  designed: 

11330 

error  occurs 

-Oat  a case  i n f or- 

• 

1 

nation  is  read 

t 

• 

from  file  ? ac  e 1 ed 

■DBASE  INPUT- 

i 

1 

-Commands  are 

• 

1 

i nout  via  t erm i - 

1 

< 

nal  only 

• 

• 

-Cutout  can  oe 

• 

• 

directed  by  user 

• 

# 

to  terminal  or 

1 

1 

file 

t 

2 

Check  Boundary 

Program  neglects 

1 

conditions  unaer 

i nou t or  orovides 

O.K. 

• 

1 

CP/CHS 

aoorooriate  error 

1 

1 

messages 

1 

• 

3 

Test  SU  func  t i ons 

a)Members  can  oe 

1 

1 

under  CP / CVS 

added  to  the  lata 

1 

1 

base 

O.K. 

• 

1 

0 ) Cat  a base  in- 

• 

formation  can  be 

• 

« 

can  be  written  on 

1 

• 

out  Du t file  such 

l 

• 

that  this  file 

• 

• 

could  be  used  as 

• 

• 

i nput  file  for 

1 

1 

subseouent  runs 

1 

1 

c ) 1 1 e»!  of  vj  c an 

t 

• 

be  examined 

1 

1 

a 

Chee  k D i agnos t i c s 

O.K  . 

1 

• 

s 

Test  Termination 

O.K. 

1 u/2Q 

of  orooram 

; 1550 
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ANNE X M 


Page  2 of  3 


TEST  PHASE  DESCRIPTION 

Project  * : A 

Test  run  » : 2 Including  2 Test  Steos 

Begin  of  Test  (day/tine)  • 05/01/1200  Ena  of  Test  (aay/time)  : 05/02/2200 

CPU  tine  for  necessary  comoiles  (in  sec.):  8,75 

a)  8.75  0)  c)  d)  e)  f)  g) 

CPU  tine  for  TEST  run  (sec)  : 12. A0 

Man  Hours  for  this  Test  run  : A.O  (including  creoaration  of 

tests) 


I ) 


TEST 

STEP 

08JECTVE 

EXPECTED  ! ACTUAL  .'ERROR 

RESULT  ! RESULT ! * 

(TOLERANCE)  ! ! 

1 • 

1 • 

Da  y 
T I--E 

COMMENTS 

ANO 

CODED  ESPG* 
Tv  o£S 

1 

Test  c O'flDa  r i son 

Coiroariso"  must  IDoes  ! 

5/01 

1)  Recora  when 

o f ranae  values 

always  be  correct Inot  ! 

1200 

error  occurs 

1 work  1 

1700 

D 3 

1 for  1 

I mo  1 enen  t a t i on 

! n e g a • ! 

of  new  coae 

t i v e ! 

necessary  to 

Jnum-  } 

h and l e all 

1 bers  1 

5/01 

cases 

• 1 

* 1 

1730 

2 

Peoeat  steo  1 

! 0.x.  ! 

5/02 

using  corrected 

• • 

1 1 

1 bOO 

version  of  orogram 

• 1 

5/02 

1 • 

1 1 

2200 

J 
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ANNEX  H 


°a«  3 of  3 


TEST  PHASE  DESCRIPTION 


Project  a : « 

Test  run  # : 3 Including  1 Test  Steps 

Begin  of  Test  (day/tine)  : 05/03/0R00  End  of  Test  (oay/ti*e)  : 05/03/1600 

CPU  tine  for  necessary  compiles  (in  sec.):  R.68 
a ) R .63  b ) c)  o)  e)  f)  g) 

CPU  tine  for  TEST  run  (3ec)  : 12. 3« 

Han  Hours  for  tnis  Test  run  : 5.0  (including  oreoaration  of 
tests) 


I ) 


TEST)  CdJECTVE 

STEP; 

1 

• 

1 

f 

1 EXPECTED  JACTUAU 
! PESULT  iPESULT 
! (TOLERANCE) 

« 1 
• 1 

ERKGP 

u 

day  ; 
TI*E! 

COMMENTS 

AND 

CODED  ERROR 
TYPES 

I JTest  oroaram  ungerlAl)  functions  ! 

5/031 

1 ) decora  enen 

IC?/C«S  using 

IsRould  »ork  t * e 1 

0«00  ! 

error  occurs 

iradars  as  data 

Isame  «av  as  for  ! 0 . x . 

J 

{members  used  >n  ! 

• 

i 

J orevious  test  ana', 

i 

i 

Ifltoug  puns  ( 

• 

• 

1 (orov i dea  t mat  ! 

i 

i 

!a'l  attribute  1 

• 

lvalue  oairs  are  1 

• 

> 

1 u i t (i i n deal  gneo  1 

5/03  1 

• 

« 

i 

• 

! 1 i mi t S ) 1 

1 1 

1 1 

1800  1 
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